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INTRODUCTION 

The first comprehensive statement of the new research method, the 
cloze procedure, and its theoretical background were introduced in 1953 
by Wilson L, Taylor in an article entitled "Cloze Procedure: A New 

Tool for Measuring Readability" in the Journalism Quarterly . During 
the past 15 years, research involving this new tool has appeared in a 
great variety of investigations. 

At the heart of the procedure is a functional unit of measurement 
called the "cloze". Just as there is an apparent tendency to "see" a 
not quite complete circle as a whole circle by "mentally closing the 
gap" and making the image conform to a familiar shape, a mutilated 
sentence is filled in by completing those words that make the finished 
language pattern conform to the intended or apparently intended 

meaning. Thus, in the examples "chickens cackle and 

quack" , or "Merry Christmas and New Year" , the test- 

taker who writes in the correct word scores one cloze unit correctly 
by completing the sentence in accord with that language pattern. 

Taylor suggests that in order to satisfactorily complete a cloze, the 
subject must first know the meanings (the patterns or symbol-meaning 



2 



relationships), the forms (patterns of letters) of most or all of the 
words Involved, and also the meanings of the given combinations of 
both in a particular sentence structure. In other words, one must 
guess what the mutilated sentence means as a whole and then complete 
its pattern accordingly. Cloze then, is defined by Taylor as a method 
for intercepting a message from a transmitter, mutilating its language 
pattern by deleting words, and administering it to receivers in such a 
way that their attempts to make the patterns whole again potentially 
yield a measure of their ability to deal with the general meaning and 
form Intended. 

The construction of a cloze readability test as outlined by Taylor , 
Includes five steps: (a) delete a number of words selected by some 

random method without any regard for the function or the meaning of 
the specific words, (b) reproduce each mutilated passage with a blank 
space of some standard length in place of every missing word, (c) give 
the mutilated passage to representative samples of the subjects in 
question, (d) ask subjects to fill one word in each blank, determining 
from the context of the remaining words what that missing word should 
be, (e) total the correct number of responses for each passage and 
consider these totals readability scor'is. In contrasting the cloze 
totals of various passages, the one with the highest score would be 
considered most readable, the second one next most readable, and so on. 

Obviously this procedure is similar to the familiar completion 
test in that the subject is Instructed to fill in the blank spaces. 

It differs from the typical sentence completion test in that the 
deleted words are not pre-evaluated and selected according to the 
relative Importance in the sentence. Obviously, the cloze procedure 
does not require an expert for either test construction or administration. 
The cloze deals with contextually Interrelated series of blanks, not 
isolated ones. Rather than dealing directly with meaning, the cloze 
repeatedly samples the extent of likeness between the language patterns 
used by the communicator and that used by the receiver. 

Initial inquiry into the nature of this procedure properly involved 
the testing of its validity in relation to readability. Taylor (1953, 
1957) and numerous other investigators (Bormuth, 1962; Gallant, 1965; 
Ruddell, 1963) using only minor modifications in Taylor's method, have 
supported the initial findings that the cloze procedure ranks passages 
at differing difficulty levels in the same order as do older readability 

formulas. 

Some of the principal intervening variables relating to the cloze 
procedure have come from Osgood's (1962) concept of "total language 
context". These include verbal factors such as grammatical skills, the 
effective use of multitudes of symbols, and non-verbal cues such as 
past experience and intelligence. 

Osgood's theory of communication suggests that redundancies and 






transitional probabilities lead to the development of dispositional 
mechanisms that play a large part in transmitting and receiving 
messages. 



Redundancy . For example, "man coming" means the same 
as the redundant statement "a man is coming this way now". 

It is suggested that the latter is more like ordinary 
English; it indicates the singular number of the subject 
three times (by "a", "man", "is"), the present tense twice 
("is coming" and "now"), and the direction of action twice 
("coming" and "this way"). Such repetitions of meaning, 
it is suggested by Taylor, make it possible to replace "is", 

"this", "way", or "now" should they be deleted. 

Transitional probabilities . Some words appear more 
frequently than others in certain patterns or sequences. 

For Instances, the theory of transitional probability 
indicates that "Merry Christmas" is a more likely 
combination than "Merry Birthday". In like manner, "please 

pass the " is more often completed by "salt" 

than by "sodium chloride" or "blow torch". Some transitions 
from one word to the next are therefore more probable than 
others. 

Habits of expression take over most of the work of translating an 
individual's meaning into an organized series of language symbols for 
transmission to others. Words appearing in sequences that fit the 
existing receiving habits of the reader can be understood with minimal 
effort. When the words appear in less familiar sequences, comprehension 
or understanding may be slower; a sufficiently improbable pattern may 
seem nonsensical. 

The justification for the random deletion method used by Taylor 
is based on the fact that if enough words are deleted, the blanks will 
come to represent proportionately all kinds of words to the extent 
that they occur. Taylor suggests that cloze scores appear to be the 
measure of the aggregate Influences of all factors which Interact to 
effect the degree of correspondence between the language patterns of 
the transmitter and those of the receiver. 

CONTEXTU^ CONSTRAINTS IN SENTENCE AND PARAGRAPH RECONSTRUCTION 

Because the problem of contextual constraint is closely tied to 
that of the theoretical consideration of the cloze procedure, two 
studies on contextual constraint will be briefly discussed here. 

Aborn, Rubensteln, and Sterling (1959) suggested four hypotheses in 
relation to contextual constraint: (a) the predictability of words 

belonging to a given class is in general Inversely related to the size 
of that class, (b) increasing the context beyond ten words between 
deletions does not seem to Increase the accuracy of word prediction 
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(the length at which context contains maximum effectiveness lies 
between five and ten words), (c) a bilaterally distributed context 
exerts greater constraint than a totally preceding or a totally 
following context of the same length, (d) when the frequency of word 
class occurrence Is more controlled, words have almost the same 
predictability In all positions In the sentence with the exception of 
final positions where predictability Is much lower. MacGlnltle (1961), 
indicated that Aborn, et al. (1959) raised the question of whether the 
constraints upon words In continuous discourse were cumulative. He 
explored the constraints within complete cloze paragraphs to determine 
the effect of differing distances between omissions. It was concluded 
that the Influence of context upon a particular word choice In English 
prose decreased rapidly as the distance of the context from the word 
Increased to five words. Beyond that point the distance between blanks 
seemed to have little effect on cloze scores. 

METHODOLOGICAL CONSIDERATIONS 

The cloze test Is constructed by deleting words according to a 
predetermined system. The subject Is given a set of Instructions, 
generally very simple In nature, which Include the following (Bormuth, 
1964c) : 

1. Write only one word In each blank. 

2. Try to fill In every blank. Don't be afraid to guess. 

3. Wrong spelling will not count against you If we can tell what 
word you meant. 

Subjects may be given a short practice exercise. To date, no empirical 
Investigations Involving children have considered the effect of 
differing test Instructions on cloze scores. Tests are normally 
untimed and are administered In the classroom. 

In scoring cloze tests, the exact word deleted Is the criterion for 
correctness most often used. However, some designs call for the 
correct scoring of responses differing from the deleted word In semantic 
meaning or grammatical Inflection. Taylor (1953) found that scores 
obtained by counting both grammatically correct synonyms (S.G.C.) and 
exact words (E.W.) deleted, were not significantly superior to scores 
obtained by using only the exact word criterion. Rankin (1957) and 
Ruddell (1963) found that S.G.C. plus E.W. scoring resulted In slightly 
Increased variances on reading comprehension test scores. All scores 
obtained by counting grammatically correct responses exhibited positive 
correlations with each other. Bormuth (1965b) studied the correlations 
between standardized reading test scores and cloze test scores obtained 
by counting words as correct If their Inflections were correct In 
context. Responses were further classified according to whether the 
stem of the response exactly matched, was synonymous with, or semantically 
unrelated to the deleted words. Multiple regression analyses of these 
data showed that scores based on responses which exactly matched the 
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deleted words in both inflections and word stems » accounted for 95 
percent of the comprehension test variance that could be predicted from 
the total set of cloze test scores. It was concluded that the most 
economical and objective method of scoring cloze tests, the exact word 
method, showed the most valid results. 

Most investigators have scored misspellings correct when the 
response was otherwise correct. Mo studies to date have investigated 
Interscorer reliability in this practice. In like manner, the 
Illegibility of written responses has not received consideration. 

VALIDITY OF CLOZE TESTS 

A substantial body of research has concerned Itself with the 
validity of the cloze procedure when used to determine the readability 
of materials designed for children. While traditional readability 
formulas have, for many years, been applied to instructional materials 
to determine the suitability of reading difficulty in relation to the 
abilities of the students for whom the material was designed, the 
formulas currently available have validities that range from .3 to .7 
(Bormuth, 1967a). Most readability formulas take into account only a 
small number of linguistic variables. Almost all formulas use 
traditional word frequencies as indicators of vocabulary difficulty. 

Some formulas also include sentence length or number of syllables per 
sentence. In any case, the variables taken into account are, in some 
respects, crude and outdated. 

Cloze tests, as noted above, take into account many variables 
affecting passage readability. Taylor (1953) has compared the 
agreement of cloze score rankings of passages of varying difficulty 
and readability rankings of the same passages by two commonly-used 
formulas (Dale and Chall, 1948 and Flesch, 1949). It is reported that 
passages were rank-ordered similarly by each technique. The superiority 
of the cloze procedure was demonstrated when passages of obvious 
difficulty could be reasonably evaluated by the cloze procedure and not 
by the readability formulas. 

Traditionally, reading comprehension ability is measured by 
allowing a subject to read a passage, and then measuring his knowledge 
of the content of that passage by his performance on multiple-choice 
items written for this purpose. Studies have reported significant 
correlation between cloze test scores and scores on specially-written 
multiple-choice comprehension tests (Bormuth, 1962; Jenklnson, 1957; 

Taylor, 1956). 

A substantial number of investigations have reported high correlations 
between cloze tests and standardized tests of reading achievement. It 
is interesting to note that correlations with cloze scores are frequently 
higher for vocabulary measures than for comprehension measures. This 
is well Illustrated by Bormuth (1967b, p. 9. See Table I). 
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Table I 

Correlations Between Cloze Readability Tests and 
Standardized Tests of Reading Achievement 



Study Subjects 



Tests 



Correlations 



Jenklnson (1957) 



High School 



Rankin (1957) 



College 



Fletcher (1959) 



College 



Hafner (1963) 

Ruddell (1963) 

(5 cloze tests) 

Weaver and Kingston 
(1963» 2 cloze tests) 



College 

Elementary 

College 



Cooperative Reading C2 

Vocabulary .78 

Level of Comprehension .73 

Diagnostic Survey 

Story Comprehension .29 

Vocabulary . 68 

Paragraph . 60 

Cooperative Reading C2 

Vocabulary .63 

Level of Comprehension .55 

Speed of Comprehension .57 

Dvorak-Van Wagenen 

Rate of Comprehension .59 



Michigan Vocabulary Profile .56 

Stanford Achievement 

Paragraph Meaning .61-. 74 

Davis Reading .25-. 51 



Green (1964) 



College Diagnostic Reading Survey .51 

Total Comprehension 
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Unlike the standardized tests, cloze measures avoid the confounding 
effect of additional text materials such as comprehension questions and 
multiple-choice responses. Through the use of cloze tests, readability 
may be determined directly on the materials In question with the subject 
population for ^Ich It Is Intended. The 75 percent comprehension 
score has a long tradition of acceptance and widespread use In current 
practice (Betts, 1946; Harris, 1962; Thorndike, 1917) . According to 
this criterion, a passage Is determined suitable for a pupil's 
Instructional use If he responds correctly to 75 percent or more of the 
Items presented related to his understanding of that passage. In some 
Investigations multiple-choice tests are constructed over the same 
passages used In cloze tests. In other Investigations, cloze tests are 
constructed from passages which have been designed for use In 
standardized reading tests. In both cases cloze scores between 40 
percent and 45 percent have been found comparable to the 75 percent 
criterion (Bormuth, 1967a). 

READING COMPREHENSION AND INFORMATION GAIN 

It Is suggested by some Investigators that reading comprehension Is 
a generalized skill, best represented by the score obtained by finding 
the difference between pretest and posttest scores over a reading 
passage. Comprehension measured In this manner has been referred to by 
Coleman as Information gain. Scores obtained In the traditional manner 
confound both knowledge acquired as a result of reading, the knowledge 
possessed before the reading of the passage and a memory factor. The 
usefulness of cloze as pretest and posttest measures of Information 
gain has been considered by Coleman (1966). His findings Indicated 
that bilateral constraint and/or the Information given In cloze tests 
was such that little Information was gained by reading the undeleted 
passage. The differences between means (Information gain) for college 
students was found Insignificant and the correlation between two cloze 
scores was .93. The few subjects used In this Investigation and the 
undisclosed nature of the passages leaves the finding open to question. 
Another researcher (Taylor, 1956) has reported significant gain 
between pretest cloze scores and posttest cloze scores over a one week 
period. 

RELIABILITY OF CLOZE TESTS 

Several studies have Investigated the reliability of cloze tests. 

It has been pointed out (Fletcher, 1959; Bormuth, 1962) that cloze 
tests frequently contain a number of very difficult and very easy 
Items which are less efficient discriminators than Items In the 
Intermediate range. This fact, according to Bormuth (1962), may 
contribute to high correlations between cloze tests and other measures. 
It may also effect test-retest, and split-half reliability statistics. 
However, these same types of difficult and easy Items In cloze tests 
may be an asset since they contribute to a test's validity with a 
variety of subjects over a wide range of difficulty levels. Skewed 
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distributions are infrequently reported when cloze te.sts are carefully 
administered . 

SUMMARY OF INVESTIGATIONS INVOLVING THE CLOZE TECHNIQUE 

Cloze tests have been used with children from the first grade 
level through high school. Subjects have also been drawn from adult 
populations such as military trainees and college students. The first 
section of this summary, Section A, is concerned with investigations 
of school-age subjects. Since age and reading ability were not 
considered as Independent variables in studies using adult subjects, 
these investigations are grouped separately in Section B. 

Section A— children 



Two studies have Involved primary (first, second, and third grade) 
children. Gallant (1965) considered the reliability and validity of 
cloze tests with first, second, and third grade children; the data 
suggest that cloze tests are appropriate for this age group (see Table 
II). A second part of the same investigation argued that sentence 
length may contribute to the variance in cloze scores for first and 
second graders. It must be noted that for first graders, the cloze 
test was modified in such a way that each deletion became a three-option, 
multiple-choice test. In order to Increase sentence lengths, unspecified 
modifications were made in a passage such that Spache Readability 
Measures were Increased approximately one-half grade. The artificiality 
of the resulting prose may have confounded the comparisons made. 

Deutsch (1964) investigated first and fifth grade subjects' cloze 
scores which were derived from deleted portions of teachers' classroom 
speech and from children's speech. The relationships of first and 
fifth grade scores to race, socio-economic status, and sex, were also 
investigated. While split-half reliability varied considerably, cloze 
scores were significantly related to I.Q. A main effect favoring 
high social status was found in addition to a social status by sex 
Interaction (favoring High S.E.S., girls) on many scores. Only one 
Instance of main effect involving race (Caucasian) was found. However, 
this difference disappeared when scores were covaried with I.Q. The 
auditory method of cloze test presentation resulted in significantly 
lower means than those derived from the written form. Deutsch states 
that the oral cloze test deserves emphasis since it more closely 
approximates the stimulus conditions of a classroom. 

While many aspects of this study appear sound, the fact that cell 
sizes in this complex be tween-subjects design drop to three subjects, 
may be cause for concern in generalizing from the reported findings. 
Training instructions used in preparing subjects for cloze test 
responses were minimal. Little consideration has been given to the 
order effects of the various cloze Instruments used, particularly with 
fifth grade subjects. It is possible that practice effects confounded 
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Author 

Dato 

Subiocti 

Ago/ 

grodo. 

nuobor 



Port I 
Grodo “ 
lot. 2nd 
3rd 

N - 273 

Gollont, 

1965 



Port II 

COM 00 

Port 1 



Clooo Toot Doscription 
typo of notoriol. difficulty 
lovolf dolotion rotOf word 



Othor Mooouroo 
nooo of inotruaontt 
typo of ocoroo 



clooo dolotod* ocoroo 



Port I 

D.R. ovory 5th word. 
Porogroph Hooding Soction* 
Motropoliton Hooding Toot. 



>ort I 

Motropoliton Achiovo- 
■ont Tooto for Grodoo 
1 & 2 . 



lot grodo Mtoriol-rMdo 
3 option* aultiplo choico. 



Motropoliton Hooding 
Toot Grodoo 3 & 4. 



Scoring~~"roooonoblo opproxi- 
■otion ogrooing in poroon ond 



tonoo. 



99 



Port 11 Portjl 

Pivo poooogoo froa oiaplifiod No othor OMOouroo uood 

folk tolo froB boool roador 

■atchod to S grodo lovol. 



Sot A: 

Spocho roadobility 1^8* 1.9* 
2.5. 2.9. 3.3 



Doaian 

iubjcct ooloction. 
oubjoct doocription. 
trootoonto. controlo 



Subjocto froa two 
ochoolo in lodford 
Indiana. 

Part 1 

Coaparo cloxo acoro 
ond otondordixod 
aooouroa. Corrolotiona 
toatad for aignificonco. 



Part 11 

Sot A and 1 cloxo 
ocoroo coaparod uoing 

aaao aubjocta ao Port 1. 

Corrolotiona tootod for 
aignificonco. 



Sot 1: 

Sontonco longth woo incroaood 
in unopocifiod aonnor to 
Spocho lovol of 2.1. 2.4( 3.0* 
3.7. 4.3. 

lot grodo aotorlol aado into o 
3 option aultiplo choice* 



Findinia 

rolationahipa. concluaiono. 
ocoroo (aain of facta, 
intoroctiono) 



Part 1 

Cloxo ocoroo rolioblo 
(aplit halvoa & S.l.) 
.90-.97. 

Stondardiiod aooauroa ond 
cloxo ocoroo ovorall 
r .65-.81 «.01) 

No oox difforoncoa. 



Part 11 

Cloxo acoro aoans highor 
for aot A than aot B 
(4.01) for firat and oocond 

gradora. 

No difforoncoa found for 
third gradora. 
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other findings* The practice of covarying I.Q. test results with cloze 
test results may be valid when individually administered intelligence 
tests are used. However, group administered tests of any kind necessitate 
the use of verbal instructions, both written and oral, which may 
confound the measurement of listening and reading comprehension skills 
(see Table III). 

An investigation by McLeod (1966) used three fiction selections 
and four prose passages with a deletion every eighth word. This 
deletion procedure was then modified when it was determined that 
certain blanks evoked ambiguous responses. These blanks were replaced 
in such a manner that only unequivocal words were deleted; the criterion 
for unequlvocallty was not stated. The tests were administered to 
children in grades three, four, five, six, and seven, using alternate 
forms of the same test. When the logarithms of estimated redundancies 
were plotted against each other, the relationships were found to be 
linear. It was concluded that for "skillful" readers the passages 
were virtually completely redundant". The practice of selecting 
cloze deletions on other than a predetermined mechanical formula seems 
questionable since a random sample of the text Involved is not 
obtained (see Table IV). 

Bormuth (1967a) demonstrated a relationship between cloze and 
multiple-choice test scores. With an adequate range of subjects and 
carefully validated multiple-choice test items, Bormuth maintained 
that a 43 percent cloze score was equivalent to a 75 percent multiple- 
choice test score when corrections were made for guessing. In like 
manner, a 50 percent cloze score was shown equivalent to a 90 percent 
multiple-choice test score (see Table IV) . 

Based on Strickland's (1962) work on the patterns of oral language, 
Ruddell (1965) wrote six passages— three from high frequency syntactical 
patterns and three from low frequency syntactical patterns. Each 
passage utilized these specific patterns in the same proportional 
frequency with which they occurred in the oral language of fourth grade 
children. High frequency passages showed significantly higher cloze 
scores than low frequency passages. It was also concluded that cloze 
scores related to Stanford Reading Test scores, educational level of 
the parents, I.Q. , and the chronological age of the subjects in the 
experiment (see Table IV). 

Bormuth (1962) has compared the scores of 150 subjects on a 
specially-made comprehension test. One of the unique features of this 
test is that only words from the passage itself, or words that are known 
by 80 percent of the fourth graders in a similar population, are used 
in the test items. Three cloze passages from literature, three from 

social studies, and three from science— one at each grade level are 

used with a population of fourth, fifth, and sixth grade subjects. 
Correlations between the comprehension tests and the cloze scores 
were found to be statistically significant (.946). Using a similar 



Tabla III Invest itatlons Involving the Gloss Technique 



Author 

Pets 

Subjects 

AisT 

grede. 

nuaber 



Close Test Description 
type of Mteriel, difficulty 
levelt deletion retSt word 
clese deleted, ecoree 



Other Meeeuree 
nsM of instruaent, 
type of ecoree 



subject selection, 
subject description, 
treetaente, controls 



Findinee 

reletionehipe, conclueione, 
■corse (aein effects, 
interectione) 



Deutech 
1964 
Grede 1 
N - 127 ' 



let Grede—verbetia eaeplee of 
first grade taechare cleeerooa 
■peach, 18 eentencee in length, 
final word deleted in each 
sentence. Sentences reed to the 
child orally. 



Lorga-Thorndike I.Q., 
level 1 end 3 Non-verbal 
fora A 



Subjects: New York City 
Public Schools. 

Three-way enalyeie of 
variance . Diaensione : 
■ociel etetue, race, end 



■ex. 



Close split helves 
reliability .33-. 84. Close 
■corse related to I.Q. 

K .01) . Main effect 
■ignificence found favoring 
Hi SES end girls (aoet 
close aeeeuree). 



Grade 5 5th Grade— 100 word verbetia 

N - 127 speech eaivl* 2 'ob fifth grede 

teechere, 20X D.R. end two 50 
word peregrephe froa fifth grede 
children*! speech. Group 1 
(62 subjects) listened to 
deleted fora, in its antiraty 
onca, and wee allowed to write 
closa word he thought deleted. 
Group 2 used conventional closa 
reeding procadura (pre-closa). 



Only one inetenca of race 
aein effect on close 

aeeeuree . 

Intarectione (Hi SES A girls) 
found eignif leant on aeny 
■cores . 



Scorinn . EH, Contextually 
correct, and Graaaacically 
correct. 
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Tahlm IV . lnv«sei|atioM Involving the Closo Teclinlquo 



Author 

Daf 

Subiocti 

AiS7 

grado. 

nuabar 



Cloaa Taat Daacription 
typo of Mtarialt difficulty 
laval. delation rata, word 
class dalatad, acoraa. 



Othar Maaauraa 
naaa of inatruaant 
typo of acoraa 



McLaod 

1966 

Grada 3-7 
N - 100 



Thraa children's fiction books* 
and four proaa pasaagaa at the 
alaaantary laval* ovary 8th 
word dolotod; on tho four 
pasaagaa (I)* dolotions which 
ovokod anbig oua roaponsoa whan 
taatod on group of toachora; 
wore roplacod by ones that did 
not (11). 



Nona 



Dalian 

subject aelaction* 
subject description* 
treatnenta* controls 



Between-aubjecta design 
2>way analyaia of 
variance. 



Findinas 

relationshipa* conclualona* 
acoraa (nain effects* 
interactions) 



. Group one acorea on four 
paaaagea higher than Group 
two score on three books. 

Meana rank ordered by grade 
levels 3 through 7. Suggests 
that eat lasted redundancy 
of paaaagea rated by two 
groups linear. 



Borauth 
(1967a) 
Grades 4-5 
N - 100 



50 deletions froa each of nine 
250 word paaaagea* every 5th 
word deleted. 



Nine 31-itaa aultiple 
choice teata* one froa 
each close paaaage. Iteas 

validated by 2 "experts". 
Itaas rejected if 
negative correlations 
between iteas score and 
total score found on 73 
subjects. 



luddall 

1965 

4th grada 
N - 131 



Six passages ware specially 

constructed using aaae 
proportional word frequencies 
as oral language. Thraa high 
frequency and three low 
frequency patterns ware equated 
(Dala-Chall) and deleted (D.R. 
1:5). Passage length 254 words. 



Otis Quick Scoring 
Mental Ability Taat* New 
Edition* Beta Fom EM. 

Stanford Achievenent 
Teat Intemadiata 1. 







Uithin-aubjacta: (order) 
close than M.C. after 

reading delated passage. 

Testa for order affects 
insignificant. 



Close and M.C. shown to be 
linear by observation. 
Regression equation shows 
close acoraa of 43X when 
corrected for guessing* 
related to M.C. score of 75Z. 
Close of 50X coaparable to 
M.C. of 90X. M.C. standard 
error of regression 6 points. 



Analyaia of Variance 

w/rapeated aeasuraa. 



Split halvas (-fS.B.) relia- 
bility .85-. 90 on close 

teata. 



Close and Stanford Reading 
Taat acoraa related (r ■ 
.61-. 72). Differences were 
found between high & low 
frequency patterns (<.01) on 
coaprehenaion acorea. 



Othar variables significantly 
related to reading acorea 
include education of parents* 
l.Q. and chronological age. 



mm 
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design, Bormuth (1964a) suggested that differences In cloze test forms 
over the same passage decreased as test length Increased* A 1:5 deletion 
rate was used on each of the 250 word cloze passages (see Table V) • 

In three studies using the same type of passages (Bormuth, 1964c, 
1965a, 1965b), cloze test scores were compared with Stanford Reading 
Tests and California Achievement Tests. Curvilinear Ity, optimum 
relations between number of subjects and number of Items, and a variety 
of scoring patterns were Investigated. A 1:5 deletion ratio was used 
In all experiments. Five different cloze tests were made over the 
same passage. It was found that no differences between test forms 
reached significance. At the word level and Independent clause level, 
an analysis of regression showed relationships to be curvilinear, 
especially at the extreme ends of the distribution. Correlations 
between linguistic variables and comprehension difficulty were found 
to be significant. In the second Investigation, the optimum relation 
of number of subjects and number of items was suggested. It was 
concluded that a combination of number of test blanks and number of 
subjects yielding a standard error of about 3.0 would yield stable 
scores. It was determined that a ratio of approximately 150 subjects 
to 50 cloze Items would be appropriate, while It was suggested that 
an experimenter should use more subjects and/or Items In actual 
practice, to allow for some margin of error. In the third Investigation, 
Bormuth suggested that an exact-word scoring criterion accounted for 
95 percent of the variance In cloze scores. While grammatically 
correct and synonymous responses correlated very highly with exact 
word scores and accounted for an Increase In mean scores, little If any 
additional discrimination power was gained by using these relatively 
Inefficient scoring criteria. It was concluded that the correlations 
Increased as a function of the similarity of the meanings of the 
responses to deleted words (see Table VI). 

In a recent Investigation, Bormuth (1968) compared oral reading 
test scores and cloze scores to California Reading Achievement Test 
scores for fourth, fifth, and sixth graders. Supporting the findings 
of previous Investigations (Bormuth, 1967a), cloze scores of 44 percent 
were found to relate to reading achievement test scores of 75 percent, 
while cloze scores of 57 percent related to 95 percent on the reading 
achievement tests. Cloze scores of 33 percent and 54 percent were found 
comparable to a 75 percent and 95 percent score on oral reading (word 
recognition) tests. Due to much greater variance In oral reading (word 
recognition) scores. It was suggested that word recognition and 
comprehension criterion scores are not comparable. Large differences 
between the relationship of cloze and oral reading scores were found 
(see Table VII). 

Schneyer (1965) compared a cloze test In which the deletion rate 
was 1:10 to scores on the California Test of Mental Maturity and the 
Gates Reading Survey. It was found that I.Q. was related significantly 
to cloze scores. No Increase in scores on the Gates Reading Survey 
resulted from practice with a series of 200-word cloze passages when 



Tahl« V Inv«itlg«tloni Involving the Clou Technique 



Author 

Date 

Sublecta 

Age/ 



Cloae Teat Deacrlption 
type of natarlal, difficulty 
level, deletion rate, word 
claaa deleted, acorea. 



Other Meuurea 
naae of Inatruaent, 
type of acorea 



Pealgn 

aubject aelactlon, 
subject description, 
treatnenta, controls 



Findings 

reUtlonahlpa, conclualonH 
acorea (Min effects. 
Interactions) 



grade, 

number 



Bomuth 
1962 

4th, Sth 4 
6th grade 
M - 160 



Nine puaages, 3 literature, 3 
social atudlee, 3 science; three 
levels of difficulty, 4.5, 5.5, 
6.5, (Dala-Chall); length waa 
270 to words. Deletion rate 
1:5. 



Comprehension teat 4 
option, multiple-choice, 
31 IteM/paaaage covering 
nukterlal given In clou 
teata. Only words from 
paaaagea appeared In teat 
Iteu or were known by 



Subjects from aMll 
nldweutern towns. 
Correlations tested for 
significance. 



AAV srAdlArA . 



Positive correlation between 
comprehension and close 
teata; significant. Item 
difficulty range .00-1.00. 
Validity of close aa a 
rudablllty measure aeeM 
likely. 



Bormuth 
(1964a) 

M - 139 
Grade 4-8 



Five 50-ltam teata made from 
a«e paaaagea auch that all 
words were deleted In one of the 
5 forM. First aentanca not 
deleted; deletion rate 1:5. Five 
subject areas tested: lltsrature 
history, geography, biological 
science, phyalul aclanca. 

20 puaagaa. 



Stanford Reading Teat 



Groups matched by 
Stanford Rudlng teat 
■una. Every subject In 
each group took one form 
of each close teat. 
Correlations tested for 
significance. 



Differences In difficulty 
among teat foru made from 
Che same passage tend to 

diminish as more ItaM are 
Included. 

Standard teat form error 
designed to aid In 
determining between-forM 
differences . 



Table VI 



Investigatione Involving the Cloze Technique 



• Author 

Data 

Sublecta 

Aia? 

grada, 

nuabar 



Cloza Taat Daacription 
typa of aatarial, difficulty 
level • delation rate, word 
class delated, scores. 



Other Measures 
naaa of instrument, 
typa of scores 



Desitn 

subject selection, 
subject description, 
treatments, controls 



Findinas 

relationships , conclusions , 
scores (main affects, 
interactions) 



Itormuth 

(1964c) 

N - 6S0 
Gradea 4-8 



20 paaaagaa, 275-300 words each 
Delation rata 1:5. Five forma 

for each passage made by start- 
ing delations with first word 
(A) through fifth word (E). 
Scoring: EW. Separata spores 
calculated for three linguistic 
variablaa: word, phrase, and 
passage. 



. Stanford heading 
Achiavamant Teat Form J 



Bormuth 
1965 (a) 

N - 695 
Grades 4-8 



20 paaaagaa, delation rata 1:5. 
Scoring: EW. Content of paaaagaa 
included: literature, geography, 
history, biological acianca, 
physical acianca. Difficulty 
laval grades 4-8 (Dale-Chall) . 
250 words in length. Testa ware 
divided about every 5th delation 
to give 10 taat lengths. 



Borwith 
1965 (b) 

N - 50 
Grada 5 6 6 



20 paaaagaa, 52 close delations, 
delation rata 1:5. Scoring cri- 
teria: Exact word (EGC), syno- 
nyms (SGC) , graamaticality (SGI) , 
unrelated (UGl), and unclasaifi- 
abla (UCR). Total of 1040 items 
considered as single taat. 



California Achiavamant 

Taat 



Stanford Achievement 
Taat: Beading 



All subjects era from 
small California school 
district. Subjects strat- 
ified into 5 reading 
ability levels on 
Stanford scores. 3X5 
analysis of variance uaad 
to determine relation of 
linguistic variablaa and 
reading ability levels. 



Small Calif, school 
district 86X caucaaion, 
lOX negro, 4Z Mexican. 
IWo close testa given 
each day for 10 days. 



Multiple regression 
analysis using reading 
achiavamant acoras as 
criterion. 



Diffarancaa between forma 
(A-E) not significant. Word 
laval and indapandant clausa 
laval of analysis: regress- 
ions found curvilinear, most; 
measures significant espec- 
ially at extrema ends of 
distribution. No F-ratio 
reached significance at 
passage laval but curvilinai 
arity is suggested. Multipla 
corralationa found between 
linguistic variables 6 com- 
prehension difficulty .51-. 



Average standard error calcv 
latad for nimber items and 
nuabar of subjects. Optimum] 
relation between number of 
subjects and iteaa suggest! 
Standard deviation of dost 
testa ranged from 4.5 to 9.H 
Aludaa to assumptions umler| 
lying Lord's formulas but 
does not discuss. 



EGC scores accounted for 95| 
of variance. GraaaMticality| 
correlated poaaitivaly witl 
comprehension scores. Corral 
ationa increased as a func-| 
tion of the similarity of 
the aaaninga of the raspon 
to the delated words. 



O 



\ 



Table VII InvestigaCions Involving Che Cloze Technique 



Author 

Data 

Subjects 

Age/ 

grade, 

nuiriter 


Cloze Teat Description 
type of material, difficulty 
level, deletion rate, word 
class delated, scores. 


Borauth 


Four 13 paragraph graded 


1967 (c) 


sequences (pra-priaar through 


Grades 4, 


adult) ware delated 1:5; two 


5, 6. 


delations foras scoring EW. 


N - 120 





Schnayer 
1965 
Grade 6 
N - 66 



200 word paaaagea from basal 
reader. DeleCion race for 
creacaanc A, 1:10. OeleCion for 
creutaenc B nouns and verba only 
(alcernaCely on average 1:10). 



ocher Measures 
naae of inaCrumenC, 
Cype of scores 



California Reading 
AchieveaenC TesC grades 
4-6. 



Two of Che four para- 
graphs were Caken as 
clo£;e Cases, two as oral 
reading exercises (scored 
SB word recognition 
exercises) . 



Design 



subject selection, 
subject description, 
creaCmenCa, controls 



All subjects froa single 
school. 



Correlations obtained and 
tested for significance. 



California Teat of 
Maturity. 

Gates Reading Survey 
Foraa 1 & 11. 



Mental 



Comprehension scores 
covaried with initial 
scores. 










O 

ERIC 



Findings 

relationships, conclusions, 
scores (main effects, 
interactions) 



Reading teat score r.-inged 
froa 2.0 to 11.3. 



1 . 



2 . 



3. 



4. 



Cloze scores of 44Z and 
57X found coaparable to 
75X and 9SX on .:oapre- 
hansion scores (C.R.T.). 
Cloze scores of 33X and 
S4X coaparable to 95X 
on oral reading (word 
recognition) scores. 

Large differences between 
the relationship of cloze 
and C.R.T. and cloze and 
oral reading scores. 
Suggested that word 
recognition and coapre- 
hanaion criterion scores 
not coaparable. 



Gain scores not significant. 
IQ related to 10th word 
deletion scores (C«01) and 
noun-verb daletioiM ( .02). 
All cloze scores and stand- 
ardized reading aeaaures 
related «.01). Two cloze 
scores related «.01). 
Suggests that noun-verb cloze 
scores aay be leas related 
to IQ than every nth cloze 
scores . 



mm 



saasamaaa 




tests were corrected and lininedlately returned to the subjects* Two 
types of deletions were used! every 10th word and noun-verb-only 
deletions (see Table VII). 

Lou than (1965) found no differences on comprehension test questions 
between cloze test scores of 7th grade students when deletions were: 

(a) every 10th word* (b) only nouns • (c) only verbs* or (d) only 
adjectives. Louthan argued that cloze tests have limited value as a 
teaching Instrument. A control group reading undeleted material, 
received superior scores on the comprehension test to groups In which 
every 10th word or only nouns, verbs, adjectives, or other modifiers, 
were deleted. However, when only prepositions and conjunctions were 
deleted, or when only pronouns or noun determiners were deleted, cloze 
scores surpassed the groups mentioned above* These data suggest that 
the so called "content word" deletions (class 1, 2, 3, 4) require 
different skills than "function words" (class 5, 6, 7) (see Table VIII). 

Bloomer (1965) compared posttest scores of groups who received: 

(a) pretest and undeleted material, (b) only deleted material, (c) a 
pretest and deleted material, and (d) only undeleted material. That 
group which did not receive a pretest and did use deleted material, 
performed significantly better than any other group. One somewhat 
questionable conclusion of this study was that anxiety caused by the 
pretest accounted for the results shown (see Table VIII). 

In a subsequent study of specially— written multiple— choice test 
scores given after 24 cloze exercises. Bloomer (1966) concluded that 
sex differences disappeared with Increasing age and that cloze tests, 
made from very easy materials were less "motivating" than materials 
more closely associated with grade level of the subjects. In a 
longitudinal study of ninth graders. Bloomer argues that cloze procedure 
Is proven relatively Ineffective In producing an Increase In reading 
comprehension scores (see Table VIII). 

Section B— adults 

The studies In this section are organized Into five groups: 

(a) the nature of contextual constraint, (b) the methodology of cloze 
procedure, (c) literary style and the cloze procedure, (d) the cloze 
procedure and Information gain, (e) the cloze procedure and oral speech. 
The studies within each section are organized chronologically. 

The na ture of contextual constraint . A study by Aborn, Rubensteln, 
and Sterling (1959) Investigated the constraint upon words attributable 
to the number of words between deletions, distribution of deletions, and 
structure of context. One word was omitted from each sentence In a way 
that yielded three treatments of sentence length, four treatments of 
position of omission, and six treatments of word class. The results 
noted In Table IX Indicate that bilaterally distributed context of 
over five and under ten words yields cloze scores of max im u m 
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T«bl« VIII Investigations Involving the Cloze Technique 



Author 

Data 
Sub J acta 

Ago/ 

grade, 

nuabar 



Cloze Taat Daacription 
type of material, difficulty 
level, deletion rate, word 
claea deleted, acorea. 



Other Meaaurea 



name of inatrument, 
type of acorea 



Deaign 



aubject selection, 
aubject description, 
treatments, controls 



Findinga 

relationships, conclusions, 
scores (main effects, 
interactions) 



Louthan 
1965 
N - 236 
Grade 7 



Bloomer 
1965 
N - 146 
Grade 8 



24 passages 500-600 words each 
one undeleted form, 7 cloze 
forms: by form class (lOX 
deletions) : 

1) every 10th word 

2) nouns 

3) verba 

4) modifiers, adjectives, 
adverbs 

5) prepositions and conjunctioni 

6) noun determiners 

7) pronouns 
Control) undeleted 



Comprehension teat: 12 
questions for each 
passage: multigfV.e choice 



Correlations tested for 
significance. 



No differences in comprehen- 
sion test scores between 
class 1, 2, 3, 4. Column 
A significantly higher (C.Ol) 
than column B: 

A B 

control 1, 2, 3, 4, 6 

5 1, 2, 3, 4 

6 1, 2, 3, 4, 5, 

control 

7 1, 2, 3, 4, 6 



Deletion rate 1:10 on five 
moat difficult passages of 
HcCall-Crabba Reading Test. 



Identical pre- and post- 
teat. 

Nature of pre-teat not 
given. Completion times 
recorded for cloze teat, 
pre- and post teats. 



Subjects drawn from 



central N.Y. 




Four groups: 




A. Pre- 


undeleted 


test 


material 


B. none 


undeleted 




material 


C. none 


deleted 




material 


D. pre- 


deleted 


test 


material 


Correlations 


tested for 


significance 


e 

f — n ^ A 



1. No differences between 
(A & D) or (B 6 C). 

2. C superior to A or B or D. 

3. A and D combined superior 
to B & C combined. 

4. D and C took longer than 
A and B. 

Argues that pretests should 
not be used. 



Bloomer 
1966 
N - 976 
5th, 7th, 
9th, nth 
grades 



Twenty-four 600 word passages, 
fifth grade difficulty. 



Special comprehension 
test: 12 question multi- 
ple-choice admin, after 
each cloze passage. 



single school district, 
correlations tested for 
significance. 



11th graders-lower acorea 
than 9th graders. Sex 
differences disappear as 
age incraasea. 
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discrimination power. 

Using thirty-six, 150-word passages ranging in difficulty from 
first grade level through technical prose, Miller and Coleman (1966) 
investigated three deletion techniques: (a) mechanical deletion rate 

of 1:5 (yielded the highest standard deviation), (b) the deletion of 
only one word in each passage (yielded the highest means), and (c) a 
unilateral constraint system In which each succeeding word was guessed 
and then revealed to the subject. Method c showed the lowest scores. 
Method c, however, revealed that sequential constraint was strong 
within sentences while little constraint was shown to cross sentence 
boundaries. Insignificant Increases in performance curves were evident 
when the context surrounding deletions exceeded 20 words. The authors 
concluded that cloze scores reliably measured readability ranging 
from first grade to adult levels (see Table IX). 

The methodology of cloze procedure . In the first comprehensive 
statement of the cloze procedure, Taylor (1953a) investigated 
methodological questions relating to the potential reliability and 
validity of the cloze tests. This research, discussed in the 
introduction to this paper, will only be summarized here. Using two 
175-word fiction passages, every nth (1:10) and random 10 percent 
mechanical deletion systems were considered, scoring both exact word 
and synonym replacement separately. It was concluded that; (a) cloze 
scores ranked passages in the same way as readability formulas, (b) the 
two deletion procedures tested, yielded reliable discrimination power 
between subjects, (c) a minimum of 35 blanks with a deletion rate of 
1:10 discriminated "better" than other less frequent deletions or 
fewer blanks per passage, (d) scoring systems other than the exact 
word scoring system yielded virtually Ideni^lcal scores. In a second 
part of the same investigation, Taylor (1953b) validated the findings 
of the first study and suggested that reading abilities of individuals 
might be assessed by utilizing the cloze method, since significant 
between-subjects F-ratlos held up across passages (see Table X) . 

Fletcher (1959) investigated the relationship between three 
250-word cloze passages (deletion rate 1:5) using an exact word 
scoring criterion. He found that: (a) there was a substantial 

positive relationship between a subject's ability to use context clues 
and ability to comprehend rapidly, (b) ability to use context clues 
was significantly related to "general intelligence", or more 
specifically to the verbal factors of "general intelligence", 

(c) appropriate cloze procedures can be developed as the basis of a 
test to measure a subject's ability to use context. It was argued 
that this measure was both valid and reliable. In addition, the 
instrument would be easy to prepare (see Table XI). 

Constraints within complete prose paragraphs were investigated 
by MacGlnltle (1961) by comparing restorations of omitted words when 
context at different distances from these words was deleted. Fifteen 












Table IX Investigations Involving the Cloze Technique 



Author 


Cloze Test DeKcciPtion 


Date 

Subjects 

Age/ 

grade, 

number 


type of material, difficulty 
level, deletion rate, word 
class deleted, scores. 


Aborn, 


Words from 1,380 six, eleven. 


et. al. 


and twenty-five word sentences 


1959 


from popular magazines were 


N - 2A 


deleted from beginning, early 


freshmen 


medial, late medial, and final 
positions. Scoring— E.W. , 
misspellings correct. 



Other Measures 



name of Initrunentf 
type of scores 




Design 



subject selection, 
subject description, 
treatnents, controls 



Selected from the top 
lOX on reading and 
English test and top 25 % 
on S.C.A.T. Tested every 
day for A weeks, 1.5 
hr s /day. 



Findings 



relationships , conclusions , 
scores (main effects. 
Interactions) 



1. Length and distribution 
of context are indepen- 
dent sources of constraints 

2. Predictability of words 
Inversely related to size 
of class. 

3. Context over ten or under 
five words produces 
maximum constraint. 

A. Bilateral context 

distribution Is optimum. 

5. When frequency or word 
class Is uncontrolled 
predictability similar 
for all deletion positions 
except final which Is 
lower. 



Hiller A 

Coleman 

1966 

N - A79 

college 

students 



Thlrty-alx 150 word NcCall- 
Crabba teat passages from 1st 
grade through difficult 
technical prose. Three dele- 
tion forms (a) D.R. 1:5 (b) 
only one word/paasage, (c) 
unilateral constraint only (each 
succeeding word guessed then 
revealed). Scoring— E.W. for 
(b) and (c); (a) E.W. weighted 
3, syn. as 2, form class as 1. 



None 



Treatments ; 

(a) 20 S's allowed 6 min/ 
passage. Order of 
presentation randomized 

(b) A50 S's three for 
each version of each 
passage, (c) 9 S's, new 
word exposed every 7 
seconds . 



Synonym and form class scores 
correlated .99 with E.W. 





Mean 


S.D. 


a) 


5A.6X 


1A.5 


b) 


63. 8X 


11.0 


c) 


33. 7X 


7.6 



Correlations: 
a with b— — .95 
a with c— — — .87 
b with c— — .87 
(C) shows sequential con- 
straint within sentences. 

(Tau for 8 positions .79, .01) 
Little constraint across sen- 
tence boundrles. From a given 
position no Increase In 
performance curve after 20 
words . 



Table X Investigations Involving tha Close Techiii(|uu 



Author 

Date 

Sublacta 

Aga/ 

grada. 

nuabar 



Closa last Description 
typo of aaterial, difficulty 
lovol* dolotion rata* word 
class dolotod* scoros. 



Other Meaoureg 



naae of instruaant* 
type of scores 



Desian 



subject salectiont 
subject descrlptiont 
traataants* controls 



Findinaa 

ralationshlps» conclusions, 
scores (aain affects, 
interactions) 



Taylor 
1953 1 
N - 24 
adults 



Three 175 word (approx.:) fiction 
passages, Flasch reading ease 

47, 68, 89; Dala-Chall 6.4, 7.1, 
9.2. Each passage delated 1:10 
and randoa lOX. Scoring E.U. 
Separata synonya score counted 
1/2 E.W. score. 




Order of presentation 
controlled. Analysis of 
variance. 



(see 1 above) 



1. Cloze scores ranked 
passages sane as Flasch 

and Daitt-Chall foraulas 
in every instance. 

2. Significant P «.01) 
between difficulty lavals. 

3. All dalation procaduras 
yialdad raliabla discria- 
ination power between 
subjects. 

4. 35 blanks, D.R. 1:10 
discrlainated "batter'* 
than other lass frequent 
dalationa or fewer blanks/ 

passage. 

5. Synonya scoring yialdad 
"virtually identical 
differentiation." 

6. Praaantation order had no 
affect on scores. 



1. Findings of II validated: 

Sons passages ranked in 
saaa order. 

2. Closa ranked passages 
aora accurately than 
Flasch or Dala-Chall. 

3. Closa rankings consistent 
acroaa sub J acta. 

4. Dalation aathod and rata 
findings inconclusive. 

5. Reading abilities of 
individuals night be 
asaaaaed ainca significant 
between subjects F scores 
held up across passages. 



Taylor 
1953 II 
N - 72 
adulta 



All passages above (Taylor, 1953 
1) and seven additional 175 
word fiction and non-fiction 

selections used. Each passage 

delated 1:5, 1:7, 1:10, lOX. 
Flasch and Dale-Chall rankings 
differed on additional passages. 



Nona 




Tabu XI 



InvastitatioM Involvlnt th« Closa T«chnlqu« 



Author 

Pato 

Subiacta 

Aia/ 

trada. 

nuabar 



Cloaa Taat Daacription 
typa of ■atorial* difficulty 
laval. dalation rata, word 
claaa dalatad* acoraa. 



Othar Maaauraa 
lUMM of inatruMnt* 
typo of acoraa 



Daaian 

aubjoct aolection« 
aubjoct doacriptioiit 
traataanta* controla 



Findinaa 

rolationehipa* concluaiona, 
acoraa (aain offocta* 
intoractiona) 



Flotchor 
1959 
N - 174 
Froahaon 



Throo 250 word* D.R. 1:5. 
Scoring— E.U. Fiction; difficul- 
ty for each paaaaga (Lorga 
Foraula) waa 5.63, 6i99( and 
6.94. 



(a) Dvorak- Van Uaganan 
Rata of Coaprbhanaion and 

(b) CooporativOf Spoad of 
Coaprbhanaion Taat* (c) 
Cooporativo Laval of 
Coaprahanaion Taat. (d) 
Cooporativo Vocabulary 
Toatt (a) Aaarican Coun- 
cil on Education Exaain- 
ation of Collogo Froah- 
aan. 



Corrolationa calculatod 



Roportad CorroUtiona: 

1. Closo and rata (apoed) 

of coaprbhanaion aoaauroa 
acoraa (a) .59 and (b) .57. 

2. Closo and lovol of 
coaprahanaion acoro (c) 

.54. 

3. Closo and vocabulary 
(d) .63. 

4. Closo and gonoral vorbal 
ability (a) .72. 



MacGinitio 
1961 
N - 600 
Collogo 
Studonta 



IVo 14b word doacriptivo 
narrativoa with Lorga roadabil- 
itioa of 4.9 (A) and 7.0 (S). 
Fiftpon difforont oaisaion sots 
in nlno pattorna. usod with oach 
pasaago: 1:3, 1:6, 1:12, 1:24* 
starting on first word; abovo 
pattorna aro ropoatod starting 
first ainus ono word, 2:6, 2:12 
2:24 starting on first word; 
2:6, 2:12, starting on first 
word alnua throo; 2:13 starting 
on first word alnua two; 4.24 
starting on tho first word. 
Scoring— E.W. 



Vocabulary tost— G. 
Fora 162. 



9 



T 



Protost (Fora 16 2) 
followad by closo 
passagos. Moan of 

Subjocts on Aaarican 
Council Psychological 
Exaa in top 25X 



1. No difforoncos wars 
significant botwoon 
closo acoraa froa toots 
using 1:6, 1:12, 1:24 
dalation ratos. 

2. A dalation rata of 1:3 
aada rostoration aoro 
difficult. 

3. Oaitting words in pairs 
aada closo tost acoraa 
lowar undar all conditions. 
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different omission sets, in nine patterns, were used with two 144-word 
descriptive narratives of sixth grade difficulty level with 600 college 
students. It was concluded that words were equally restorable every 
24th, 12th, or 6th word, but that third-word omissions made restoration 
significantly more difficult. It was further concluded that the 
omission of two adjacent words Increased difficulty but that the two 
words were equally restorable. The Influence of context upon word 
choice, it was suggested, decreases rapidly with the distance of the 
context from the deletion up to about five words. Distance beyond 
five words, it was argued, has less effect on cloze scores (see Table 
XI). 



Weaver and Kingston (1963) conducted a factor analyses of cloze 
procedures and other measures of reading ability. Contrary to previous 
investigations, it was concluded that the relationship of cloze tests 
to standard tests used in this study was only moderate. By means of a 
rotated factor analysis, three factors were extracted from the 18 
tests used which were identified as verbal comprehension, redundancy 
utilization, and (more tentatively) rote memory. The authors point out 
that there is much specific variance connected with cloze test scores 
that, in their investigation, was not accounted for by standardized 
tests of reading comprehension, listening, and language symbol 
manipulation (see Table XII). 

Flllenbaum (1963) obtained cloze scores from transcripts of speech 
from Thematic Apperception Test responses using varying deletion rates 
(1:2, 1:3, 1:4, 1:5, 1:6); 200 deletions were made. Cloze responses 
were considered as correct if they were verbatim or from the same form 
class as verbatim responses. The relation between these two measures 
was examined. The main findings of this investigation were that cloze 
scores Increased moderately with decreasing density of deletions. It 
was argued that the determinants of form class and verbatim items in 
cloze completions are different since form class predictability is more 
dependent upon the immediate grammatical environment whereas verbatim 
predictability depends upon both this factor and remote topical content 
or semantic features of discourse. It was also suggested that these 
determinants vary with the particular grammatical class under 
consideration (see Table XIII). 

An investigation by Hafner (1963), of different methods of scoring 
cloze tests, considered five different measures: (a) connective words 

only, (b) content x^ords, (c) connective /content quotient, (d) grammatically 
correct but lexically Incorrect responses, and (e) Incongruent scores 
(responses which are contrary to clearly stated information In the 
context or language pattern of the sentence in question). These 
measures were compared with three standardized measures, two experimental 
tests, and course grades in reading methods for college seniors. The 
major findings of this investigation were that cloze scores correlated 
positively and significantly with all the standardized measures. The 
Incongruency measure correlated negatively and significantly with 



Table XII liivestigatlons Involving the Cloze Technique 



A uthor 

l >ate 

Subiecte 

Age/ 

grade, 

(lumber 



Weaver & 
Kingston 
1962 
^ •> 160 
College 
Juniors 



C loze Teat Description 
type of material, difficulty 
level, deletion rate, word 
class deleted, scores* 



otiter Measures 



Eight AO Item cloze 
included: 

Printed passages: 
Materials 

a) Essay 

b) Essay 

c) Speech 

d) Speecit 

Spoken passages: 
Materials 

a) Essay 

f) Speech 

g) Essay 

h) Speech 



tests 



Oelfc lions 
Mechanical 
Form class 
Mechanical 
Form class 



Deletions 
Mechanical 
Form class 
Form class 
Mechanical 



Design 



name of instrument, 
type of scores 



subject selection, 
subject description, 
treatments, controls 



L. 






Davis Reading TchC 
Modern Language 
Aptitude Tests : 

Number Learning 
Phonetic Script 
Spelling Clues 
Words in Sentences 
Paired Associates 
STEP Listening Test 
Ohio State Psycholog- 
ical Exam 
Vocabulary 
Word Relations 
Reading Comprehension 



TO»in subjects' design, 
no contt(»l lor order 
effects mentioned* 30 
minutes allowed for each 
cloze teat (written or 
oral). Listening clozu 
tests; subjects heard 
entire passage first 
then only context pro- 
ceeding deletion* 

Correlations calculated 
and rotated factor 
analysis for 1 Reading 
Comprehension 
XI Cloze Factor 
III Memory and analogy 



Findings 

relat lonsh Ips , cone lus Ions , 
scores (main effects, 
interact Ions) 



Relationships with cloze 
scores on spoken passages 
accounted for 7D% of 
signifit*unt relat lonsliips 
over **(). No cori'i.- lotion 
with close scores exceeded 
.61. Relatlonehlps cloze 
scores and Ohio State 
Psychological Examinations 
accounted for 70% of the 
correlations over *50. The 
most heavily loaded factor 
Is reading comprehension 
(24.83) followed by (3.93) 
and memory and analogy 

Cloze teats were found only 
moderately related to verbal 
comprehension. Suggests that 
3 n aptitude distinct from 
verbal comprehension under- 
lies cloze scores on 
printed materials. Much 
specific variance connected 
with cloze scores unaccounte 
for* 



Table XIll Investigations Involving the Cloze Tcclmique 



Author 


Cloze Teat Description 


Other Measures 


DcRign 


Data 

Subjects 

Age/ 

grade, 

number 


type of material, difficulty 
level, deletion rate, word 
class delated, scores. 


name of instrument, 
type of scores 


subject selection, 
subject description, 
treatments, controls 


Fillanbaum, 


Transcript of speech from T.A.T. 


Nona 


Cloza scores considered 


at al. 


rasponsas. Deletion rates 1:2, 




by graamacical class. 


1963 


1:3, 1:4, 1:5, 1:6. Two hundrad 




Subjects tasted in groups 


N - ? 
undargrad 


delationa made at each rate. 
Scoring— E.U. 




of 5 to 20. 

Order of teat praaantatioi 
randomized. 


Hafnar 


One 250-word article on educa- 


1. Michigan Vocabulary 


Correlations tested for 


1963 


tional television. O.R. 1:5. 


Profile 


significance. 


N - 32 


Scoring: 


2. Otis Quick Scoring 




collage 


A. Cloze K.U. 


Mantel Ability Teat, 




seniors 


1. Incorrect but correct form 
class (G.C.I.A.) 

C. Connectives correct 

D. Connective /Content Quotient 

E. Contrary to contextual 
information or language 
pattern (Incongruent) 

F. Cloza Speed 


Gamma Fa. 

3. Uaachaaler-Bellavie 
Intelligence Scala, 
Information (sub. T) 

4. Hefner Intelligence 
Teat (experimental) 

5. Hafnar General 
Information Teat 
(experimental) 

6. Course Grade 








(Reading Mathoda) 





Findings 

relationships, conclusions, 
scores (main effects, 
interactions) 



Proportion of verbatim 
completions: 

Content Function Total 
words words 



1:2 


.115 


.483 


.323 


1:3 


.245 


.581 


.434 


1:4 


.297 


.606 


.464 


1:5 


.319 


.646 


.514 


1:6 


.321 


.704 


.540 



Cloze scores related to 
intelligence, vocabulary, 
information and course ouirks. 
Cloze spaed related to infor- 
mation, achievement, vocabu- 
lary and intelligence. Cloze 
variables did not enter a 
Multiple R prediction of 
collage grade point average. 
Relations «.01) (*<.05). 





A 


H 


C 


D 


K 


F 


1 


.56 


n7s 


n7s 


.-42 


.-50 


.48 


2 


.73 


.68 


.69 


N/S 


.-48 


.64 


3 


.56 


N/S 


N/S 


N/S 


N/S 


N/S 


4 


.46 


N/S 


N/S 


N/S 


N/S 


N/S 


5 


N/S 


(42 


N/S 


N/S 


.-47 


?41 


6 


.65 


.47 


N/S 


N/S 


.-54 


?41 
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vocabulary intelligence measures and course grades. It is hypothesized 
that the ability to complete blanks where function words have been 
deleted was an indication of the ability of the subject to conceptualize 
interrelationships among ideas. Personality variables were also 
considered in this investigation (see Table Kill). 

In a study by Musgrave (1963) a novel s ;ory was presented to 200 
subjects. In one form, the story was presented with information on 
plot and character; in another form the story was presented without the 
information (which was contained in a paragraph preceding the story 
Itself). Cloze scores were derived from both exact word scoring and 
''commonality scoring" (defined by the author as words which are exact 
duplicates of the most popular responses made by the group of subjects 
under consideration) . It was found that exact word and commonality 
scores correlated highly with each other and that there were no 
significant differences in those passages preceded by "who" explanations 
or "what" explanations (see Table XIV). As was suggested by the author, 
however, these findings may have been confounded by a crucial variable, 
the topic of the story itself. If, as Coleman suggests, contextual 
constraint is more powerful in determining cloze completions than is 
prior Information, a successful demonstration of the effect of these 
context factors would not be possible. 

Luke (1964) investigated deletions (10 percent) in 150-word fiction 
passages. These deletions were of four types: nouns only, verbs only, 
adjectives only, and combinations of the three. In this investigation 
the mean scores for verb deletions were higher than combination scores — 
which were in turn higher than either nouns or adjective means. The 
seeming contradiction of Louthan's (1965) data may lie in passage 
differences or in the age of subjects (7th grade vs. adult) (see Table 
XIV). 



In another investigation of different types of deletion methods, 
Greene (1965), using a 600-word passage, deleted only content words in 
a mechanical 1:12 manner and in an 8 percent mutilation method. The 
content word tests showed higher scores than those tests that were 
mechanically deleted. The author points out that while this modified 
cloze procedure did produce higher means, test construction time is 
greatly Increased and there is a loss of objectivity in item construction. 
It may also be pointed out that rational deletion ceases to measure 
passage difficulty since the deletions are no longer representative of 
the population of possible deletions within the sample (see Table XV) . 

It may be noted that an investigation by McLeod (1965) has resulted in 
the publication of a reading test (GAP Reading Comprehension Test, 

1965) based on a cloze type procedure using rational deletion of certain 
words. The instrument is designed for use with children of upper 
elementary school age. 

Literary style and cloze scores . Using Miller and Selfridge's 
(1950) 50-word passages (rearranged in eight orders from highest to 



Tabit! XLV lavestigations Involving the Cloze Technique 



Author 


Cloze Test Description 


Other Measures 


Design 


Date 

Subjects 

Age/ 

grade, 

number 


type of material, difficulty 
level, deletion rate, word 
class deleted, scores. 


name of instrument, 
type of scores 


subject selection, 
subject description, 
treatments, controls 


Musgrave 


Three 200 word newspaper stories 


> None 


Treatments 


1963 
N > 200 
undargrad 


D.R. lOZ random. Scoring— E.W. 
and "commonality". 




I. Only simple instruc- 
tions & deleted 
passage. 

11. Same as I but first 
para not deleted 
(told "who" and 
"what") . 

III. Same as II but only 
"who" portion of 1st 
para included. 

IV. Sane as 11 but only 
"what" portion of 1st 
para included. 








Analysis of variance. 


Luka 
1964 
N > 24 

female 

undargrad 


lOZ of four 150 word fiction 
passages were deleted four ways: 

a) nouns only 

b) verbs only 

c) adjectives only 

d) combination (5 each) of the 
3 form classes above. 


None 


Each of the four deletion 
types were administered 
so that no subject 
received two cloze tests 
over the same paslsage. 

Only percentages reported 




Scoring— B.U. 







Findings 

relationships, conclusions, 
scores (main effects, 
interactions) 



Mean for ''connonality'* 
scores ranged from 7.92 - 
8.76 out of 20 while E.W. 
means were 6.04-6.54. 
Treatment II and Treatment 
III means somewhat higher 
than IV or I for both 
"commonality*' and E.W. 
scoring criteria. No 
significant main effects 
or interactions. 



Higher cloze scores reported 
for verbs than nouns or ad- 
jectives (M 6.12). Combina- 
tion scores (M 5.29) were 
followed by nouns (M 5.00) 
and adjectives (M 4.20). 



Table XV 



Investigations Involving the Cloze Technique 



Author 

Date 

Subieef 

AgS7 

Rr«de, 

number 



Cloze Teet Deecrintion 
type of material, difficulty 
level, deletion rate, word 
class deleted, scores. 



Other Measures 
nasie of Instrument, 
type of scores 



Desian 

subject selection, 
subject description, 
treatments, controls 



Findings 

relationships, conclusions, 
scores (main effects, interactions) 



Greene 

1965 

M-128 

Upper 

clasimen 

& grad. 

. students 



One 600 word post cloze passage. 
Form B-C deleted 1:12; Form A-D 
8X deletion, content words only. 



None 



65 S's took B-C, Split half reliability of B-C (.523) 

63 S's A-D and A-D (.757) significant (<.05). 

Item discrimination (r over .40) greata 
for A-D (64%) than D-C (55%). Modified 
cloze (A-D) produced higher percentage 
of "effective" items. 13% of items In 
B-C extremely difficult. 
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lowest statistical approximation to English), Salzlnger, Portnoy, and 
Feldman (1962) found that the proportion of words guessed correctly In 
any grammatical category Increased as the approximation to English word 
order Increased. In this study, which Is highly related to communication 
theory, Salzlnger at al. concluded that the assumption of equal Intervals 
between successive orders of approximation Is untenable when using cloze 
scores as a criterion. The results further Indicated that the 
relationship between memory and order of approximation can be explained 
In part by the syntactical structures themselves and In part by meaning 
called for by the context, (see Table XVI). 

Coleman has Investigated two aspects of the structural characteristics 
of passages and, with the use of the cloze technique, has evaluated 
their relative readability. In one study (Coleman, 1962) , three 232" 
word passages from adult non-flctlon were rewritten so that sentence 
len3th varied from 15 words to about 38 words. Subjects were matched 
In groups of three and each was Instructed to read one of the passages. 

It was found that, while Increased sentence length accompanied lower 
cloze scores for that sentence, differences were small. These results 
Indicate, according to Coleman, that the Indiscriminate break up of all 
long sentences Into short ones will Improve the passage either 
slightly or not at all (see Table XVI). In a second Investigation 
(Coleman and Blumenfeld, 1963), two passages from a psychology text 
and ten selections from technical writings were presented to subjects 
In both nomlnallzed and active verb forms. Cloze tests were composed 
using a Is 5 deletion rate and each form was given to half of the 
subjects. Function words and content word scores were analyzed 
separately. Differences In cloze scores favored the active verb form. 
Content word scores for the active verb form were superior to content 
word scores for the nomlnallzed sentences. It was concluded that the 
word patterns of the material written In active verb form were more 
predictable, thus allowing the Individual, who Is required to read and 
gain Information from a passage, to know relatively more about the 
material after the first reading. It was suggested that these results 
have Implications for preparing teaching material (see Table XVI). 

Without specifically defining the characteristics of the passages 
used, Bormuth (1965c) has compared the cloze scores on two author's 
material with the scores that subjects received on tests of ability to 
detect literary style. Correlations with the style test were 
significant. However, no differences between correlations for the two 
passages approached significance. Bormuth seriously questions Weaver 
and Kingston's (1963) conclusions and suggests that because they, like 
Rankin, have used a selective deletion procedure, factor analyses fall 
to load heavily or uniformly on the same factor (see Table XVII). 

Darnell (1963) compared the cloze scores of seven binary transpo- 
sitions of a 240-word passage. It was logically determined that the 
seven degrees of disorganization resulted In a loss of clarity. The 
findings showed that cloze scores ranked the passages In a manner that 






Table XVI Investigations Involving the Cloze Technique 



B|luthor 


Cloze Test Description 


Other Measures 


Desisn 


Findings 


bate 

■Sublects 

bge/ 

igrade. 

munber 


type of material, difficulty 
level, deletion rate, word 
class deleted, scores* 


name of instrument, 
type of scores 


subject selection, 
subject description, 
treatments, controls 


relationships, conclusions, 
scores (main effects, interactions) 


Isalzinger 
let. al. 
|l962 
In-93 

9 undergrad . 


Miller and Selfridge's (1950) 
nine 50-word passages in each of 
eight orders of statistical 
approximation to English were 
deleted 1:5. Scoring— E.W. and 
form class* 


None 


Between subjects 
design. 


1. Subjects guessed a greater 
proportion of words the higher 
the order of approximutiun to 
English. 

2. Proportion of words guessed in 
the correct gramnatical category 
increased most at the end of 
orders (0-3) furthest from English, 


9 Coleman 
1 1962 
1 N-90 
Undergrad 

‘ 

l- 


Three 232 word passages from 
adult non-fiction. Each re- 
written to give three sentence 
lengths : 

Group 

A 15.4 words 

B 23.2 words 

C 38.7 words 

A cloze test was made over 
original passage. 


None 


Each subject read 
one passage in each 
group, but each read 
different group- 
passage combinations. 


Mean scores 


A - 22.4 B - 21.3 C - 20.9 

Linearity significant «.05) 

The degree by which the cloze scores 
of A & B exceeded C correlated with 
length of C. r.383 «.05) 

Argues that indiscriminate shortening 
of sentences may not necessarily 
improve the passage. 


Coleman & 
Bltnnen- 
feld 
1963 
N-lOO 
freshmen 


Two paragraphs from a psychology 
text and ten sentences selected 
from technical writings, were 
written in nominalized and 
active-verb form and divided 
evenly among the subjects. The 
367 words were deleted (D.R. 1:5' 
in ten forms so that each word 
was scored for nominalized and 
active verb scores. Scoring— 
E.W. and inflected forms correct. 


None 


Each subject received 
half nominalized and 
half active-verb 
sentences. Analyzed 
for function word and 
content word scores. 


Active-verb sentence cloze scores 
superior to nominalized cloze scores 
«.01). Content word scores for 
active-verb superior to those for 
nominalized sentences (<*01). 







I 




iWiiltlimi 






\ 



Table XVII 



Investigations Involving the Cloze Technique 



Author 

Date 

Subjects 

Age/ 

grade, 

number 



Cloze Test Description 
type of material, difficulty 
level, deletion rate, word 
class deleted, scores. 



Bormuth, 

et. al. 

1965c 

N-150 

female 

college 

students 



Two 250 word selections from 
Trollope and two from Steinbeck 
D. R. 1:5, scoring»E.W. 



Other Measures 
name of instrument, 
type of scores 



Design 

subject selection, 
subject description, 
treatments, controls 



Test of ability to 
. detect literary 

style (unidentified^ 



First cloze test 
followed by reading 



1 . 



and discussion of 
passage followed by 
second cloze test 
and test of ability 
to detect literary 
style. 



2 . 

3. 

4. 



5. 



Findings 

relationships, conclusions, 
scores (main effects, interactions) 



Cloze test reliability (split half 
and S.3.) .76 - .94. 

Correlation with style test 
.418 - .660 «.01). 

No differences between correlations 
for Trollope and Steinbeck approached 
significance. 

Study of undeleted passages does not 
seem to effect scores on measures 
to detect literary style. 

Pre-cloze and post-cloze seem to be 
equally valid. 



Darnell 

1963 

N-140 

Undergrac 



One 240 word control passage and 
one similar passage in which 
sentences are rearranged in 
seven degrees disorder, 1 
("right") to 7 (most disordered), 



None 



Niasber of binary transpositions 
for each form: 



1 0 

2 7 

3 13 

4 20 

5 27 

6 35 

7 43 



7 experimental 
groups (20 S's ea.) 
took the control 
cloze test and a 
cloze test, over one 
of the seven trans- 
posed passages. 



Seven forms were significantly 
different «.05). 



Group 


1 


2 


3 


4 


5 


6 


7 


Control 

Mean 


23 


22 


22 


25 


22 


23 


21 


Exper. 

Mean 


21 


19 


17 


18 


18 


18 


15 


Cloze 7. 
Exper. 


45 


40 


37 


39 


38 


38 


32 



1. Disorder can affect comprehension 
adversely. 



2. Amount of loss of clarity becomes 
greater as the degree of dis- 
organization Increases. 



I 
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reflected the amount of disorganization caused by the transpositions. 
Darnell concluded that disorder can affect comprehension adversely and 
that the amount of clarity lost Increases as disorganization Increases. 
It is suggested that one strength of the cloze procedure is that it is 
sensitive to the organization variable and can be used in further 
studies of the effect of word and sentence order on comprehension (see 
Table XVII). 



Cloze scores as a measure of knowledge or Information gain . 

Taylor (1956a) gave a group of Air Force trainees a cloze test, then 
allowed them to read the passage in the undeleted form. This was 
followed by another form of the same cloze test. VHille the pretest and 
posttest scores correlated highly, the gain was significant. In the 
same study, Taylor investigated a methodological question and, by 
comparing function word scores and content word scores (passages 
deleted 1:5), he found that function word deletions produced higher 
means but showed smaller gain scores than content words. Correlations 
between cloze and comprehension test gain scores were highly significant 
for contentlves. However, because of the high correlations between both 
content and function word scores, Taylor suggests that there is little 
advantage in preclassifying words and limiting deletions to them, and 
no advantage in putting one's self to the trouble of judging and scoring 
~syndfiym8i — in this aame investigation, Taylor concluded that about 50 
blanks were the chances of mechanically selecting 

easy and hard words to cancel outand~~yl€td~-»-stablfi_.§cor^of the 
difficulty of the passage or the performance of an Indlvloua 
specific words of extreme ease or difficulty (see Table XVIII). 



The findings of Bloomer (1962) suggested that when a standardized 
Instrument was used to measure possible gains from cloze test completion 
that gains, if any, were not significant. Bloomer reported that one 
control group was completely eliminated because the volunteer subjects 
failed to complete the experiment. This suggests that the findings 
from the other groups Involved may be affected by the manner in which 
subjects were selected (see Table XIX). 



Coleman (1966) used a pre-cloze test and a post-cloze test as 
measures of information gain. While certain gains were reported, none 
of statistical significance was found. Coleman speculated that this 
was due to the nature of the cloze as a pretest measure by suggesting 
that the deleted paragraph contains substantial information (see Table 
XIX). 



The effectiveness of oral communication as measured by cloze scores . 
Dickens and Williams (1964) compared scores on the American College 
English Abilities Test with scores from an orally administered cloze 
test. It was indicated that cloze procedure was a useful technique for 
measuring the comprehension of oral messages. Correlations with the 
English Abilities Test were significant and it was suggested that the 
cloze test taken orally was both reliable and valid. Oral cloze 



Table XVIII Inveitlgatlon* Involving the Cloise Technique 



Author 

Date 

SubUcti 

Age/ 

grade, 

nunbar 



Taylor 
1956 I 

N-68 

Air Force 

Trainaaa 



Taylor 
1956 II 
N-106 
Air Force 

Trainaaa 



Cloze Teat Daacription 



type of BMtarial, difficulty 
level, deletion rata, word 
claaa delated, acoraa. 



Eight 9 line aaavlaa of an Air 
Force aupply nanual ware deleted 
(D.R. 1:5 iapliad). 



Siailar to Taylor 1956 I but 
delated only functora or 
content ivea. 



Other Meaaurea 



name of inatrument, 
type of acorea 



Design 

aubject aelection, 
subject description, 
traatamnts, controls 



Specially conatructai 
comprahanaion testa 
with reliability 
"checks" (no data) 

and item analyaaa 

(no data). Fra & 
post teat forma used 
Air Forces Qualifi- 
cation Teat also 
compared. 



None 



Subjects given one 
dote form, than 
prataat ona. One 
weak later given 
undalatad articla 
to read followed 
by the post teat & 
another copy of the 
dote form. 
Correlations. 



(see above) 



Findings 



ralationships, conclusions, 
acoraa (main affects, interactions) 



1. Prataat acoraa corralatad .70 and 
post tast acoraa corralatad .80 
with each other. 

2. Frataat-poat teat relation .83 
(comprahanaion) and .88 (dote). 

3. Comprahanaion acoraa corralatad .65 
(before) and .70 (aftar) with 
A.F.Q.T. 



ilatad .73 and .74 



4. Cloia acoraa i 
with A.F.Q.T. 



5. Haan gain 8.5 points for doia and 
4.8 for comprahanaion «.001). 



Y, Cain ido r a. algiiilieant f a r b o th 

functora and contantivea. 

2. Functora produced higher amana but 
amaller gain acoraa than contantivea 
acoraa. 

3. Contantivea showed lass stability 
than machanical delation procedures. 

4. Correlations between dote and 
coaqirahanaion taata gain acoraa 
was .80 for contantivea only and 
mechanical deletions. 




T«bl« XIX Inv«itlg«tlon8 Involving Close Technique 



Pete 

gubiecti 

Age7 

grade, 

nuabar 



Close Taet Deicription 
type of Material, difficulty 
level, deletion rate, word 
class deleted, scores* 



ocher Measures 
nasw of instruaent, 
type of scores 



Desieu 

subject selection, 
subject description, 
treatMcnts, controls 



Gindinas 

relationships, conclusions, 
scores (tsain effects, interactions) 



Blooner 

1962 

N-88 

college 

students 



500 word passages froa each of 
seven levels of a coasarcial 
science and social studies se- 
ries* D*R* 1:10 



Diagnostic Survey 
Test Fora A & B 



Control group 1 used 
Gilbert's Power and 
Speed in Reading 
Control. 

Control Group 11 
took pre & post 
tests (Diagnostic 
Survey) only. 

Bxperiaental group 
took pre test, post 
tests and close 



All but 2 subjects in control group 
1 dropped out* 

2* While gain scores for both control 
group 11 and the experlaental group 
were significant no between group 
differences were significant* 

3, Total and coapreheneion scores 

favored the experlaental group but 
vocabulary asasure favored control 
group 11 (non-slgnlflcant). 



exercises* 



Coleaan 

1966a 

N-20 

Undargrad 



Thirty-six 150 word McCall-Cnbbs 

test passages froa first grade 

through difficult prose. 
Scoring— B.W* , D.R* 1:5* 



Nona 



infomation Gain for 

each passage is 

defined as diffaracc 
between pre-close 
and post»close 
scores* 



Mean gain 1*85. No significance, it 
contended is due to nature of close 
as a pre-test* 






is 








S's took close test 
than read undeleted 

passages than 

another foia of 
close test* 



mm 



am 
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procedure may have aome advantagea over multiple-choice teata aa a 
reaearch technique for studying the comprehension of spoken messages. 
DeVito (1965) obtained cloze scores on samples of oral and written 
discourse by skillful communicators. It was suggested that the 
passages used did not differ in overall comprehensibility, in 
vocabulary load, in sentence structure, or density of ideas. No 
differences between groups was reported. It may be that the close 
procedure is not a sensitive device when vocabulary and other 
characteristics are relatively advanced and the respondents are 
comparatively less able (see Table XX). 



SUMMARY AND CONCLUSIONS 
THE SUMMARY— NATURE OF CLOZE 

Cloze measurement appears applicable to many types of communic/ition. 

It seems to discriminate among the "readability" levels of passages and 
among the reading comprehension levels of readers. The cloze technique 
provides a measure of the degree of correspondence between the language 
habits of the transmitter and those of the receiver. 

Although the cloze procedure measures, in a sense, the degree to 
which the transmitter and the receiver share the same structural and 
lexical communication patterns, the "total stimulus complex" of each 
may be quite different at any specific point in the communication. The 
transmitter is combining words both on the basis of previous experience 
and on the basis of the goal toward which his writing is aimed. The 
receiver, on the other hand, may lack a foreknowledge of this goal, 
thus the nature of his receptions tends to be limited only by his own 
l>«st experiencar — — 

It has been suggested (Taylor, 1956) that pairs of words that have 
a high or a low probability of occurring together, greatly influence 
cloze scores. The most frequent sequential associations would tend to 
limit the population of possible cloze responses. Thus, high probability 
messages would withstand mutilation better than low probability ones. 

While this suggests that the most readable message is, therefore, one 
that contains the highest possible number of high probability associations, 
it may be that under these conditions little or no new information is 
transferred and no novel phrase or concept would be presented. One may 
ask the question then: "Why should such material be read at all?" 

Some degree of redundancy must exist, however, if the receiver is 
to understand the message being tansmitted. A complete lack of 
redundancy would tend to create total disorganization. It has been ‘ 
shown that highly disorganized messages have little or no communication 
value (Sal Zinger, 1962; Darnell, 1963). 

The sunBiary of studies of children. Studies of children using the 




Table XX Investltatloni Involving tho Cloze Technique 



Author 


Cloze Test Description 


Other Maaajiroa 


Dee ten 

subject selection, 
subject description, 
treatswnta, controls 


Findlnsa 


Dat£ 

Sub facts 

Aga? 

grada. 

number 


type 0 ^ suitor ialf Jifficulty 
lavalf dalotion rata, vord 
class delated, acoraa. 


nasw of inatmeMnt, 
type of acoraa 


ralationahipa, conclusions, 
acoraa (suin effects, intaravtluns) 


Dickons & 

HllllasM 

196A 

H-2S3 

undorgrad 


TWO 528 word apeachaa froa tho 
8TIP* Salaction.A-ona vord 
sutilated every 5 aaconda* 
Saloction B-ona vord sutilatad 
as in section A and 5 second 
silence for roaponaaa at and of 
each aantanca* 


Aswrican Collage 
Tasting Progran, 
ingliah Abilitiaa 
Teat* 


Group 1 (M*127) 
heard close passage 

as delated in its 
entirety, than on 
second hearing 
responded* Group 11 
(M-126) heard 
sutilatad tape 
followed by 32 
sutipla choice 
quaationa taken in 
part froB STEP 
Liataning Teat and 
in part frcu inves- 
tigator 'a work* 
Correlations tasted 
for significance* 


1* Close acoraa correlated with Knglish 
abilitiaa acoraa *49 & *52 «.01)* 

2* TWO paaaagaa close scores related 
to each other *73 (<*01)* 

3* Multiple choice teat scores 
correlated with Ingliah Ability 
acoraa *48 6 *50 (<*01)* 

4* TWO paaaagaa aultipla choice acoraa 
related to each other *37 «.01)* 

5* Close teat split half reliability 
*70 4 *80* 


DeVito 

1965 

N-6S 

16-18 yr. 
olds 


Four 300 word paaaagaa (2 oral, 
2 written), D*R* 1:5* 
Bcoring— BcW* 


Nona 


Groups of 16-18 
Subjects took each 

passage* 


Pairs of asBvlaa of oral and written 
ditcouraa by skillful cosanmicatora 
do not differ significantly in overall 
coaqirahana ib i 1 i ty * 

Close acoraa 

naan S*D* 

Oral 19*84 3148 

Written 19*86 3*42 
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cloze technique began in the early 1960's and were first reported in 
1962 (Bormuth). Sources of the passages selected for deletion Included 
standardized tests, conversation of adults and children, fiction and 
non-flctlon selections from authors other than the investigator, and 
paragraphs specially constructed by the investigator. Research involving 
the cloze procedure has been used with first grade through high school 
age subjects. A modification of the standard cloze procedure appears 
mandatory at the first grade level due to the subjects' limited writing 
ability. However, from second or third grade level up the cloze test 
appears to be an effective measure. The length of the passages used 
was usually 250 words, although a few investigations Involved passages 
of more than 400 words. An optimum sample size study (Bormuth, 1965a) 
seems to establish an empirical basis for the selection of passage 
length. 

Several deletion methods have been used. The most often used is 
a mechanical every-nth procedure in which one word in every five is 
replaced by a blank of 15 spaces. While this procedure appears to be 
entirely adequate for the purposes of measuring passage difficulty or 
"readability", the best method for determining the reading comprehension 
level of individual subjects has yet to be established. Other deletion 
methods have included 10 percent and 20 percent random deletions, 
deletions based on form class, and deletions based on a rational 
selection of those words which seem, in the opinion of the investigator, 
to be critical to the meaning transmitted. 

In the vast majority of studies, the equating of passages was 
based on ratings by readability formula. The Dale-Chall formula was 
used most frequently. In several studies a publisher's estimation of 
passage difficulty was reported. Presentation methods seemed to be 
of two distinct types. The pre-cloze written method was most frequently 
used with large groups. However, in a few investigations an oral 
presentation was used in which the subject listened to the passage in 
its entirety prior to the cloze exercise. 

Other instruments administered to validate cloze results included 
standardized achievement or intelligence tests and, in several 
instances, specially constructed multiple-choice items. These 
questions, based on information contained in the passages over which 
the cloze test was made, were carefully validated in only a few 
instances. 

The subject population "represented" a variety of racial and 
socio-economic backgrounds. Hbwever, many investigators made no 
attempt to describe the socio-economic, racial or ethnic characteristics 
of the population. Frequently the bases for descriptive comments such 
£is "representative" were not discussed. While one study involved in 
excess of 900 children, the great majority considered the responses of 
one or two hundred subjects. These were, in many instances, drawn 
from a single school or a single school district. The selection of the 



38 



school or district may have been predicated on administrative 
considerations . 

The exact word-response criterion was most often used. Several 
investigations used alternative scoring criteria such as synonyms, 
form class similarity, or "grammaticality" . The comparison of cloze 
scores with standard scores seemed to hold up well and was reported 
significant in virtually all studies. Most investigations with 
children Involve methodological types of Independent variables j only a 
few studies have used the cloze procedure as a teaching technique. The 
value of cloze under these conditions has yet to be demonstrated. 

The summary of studies of adults . Of the 23 adult studies 
reported, the substantial majority are methodological investigations. 
Cloze scores, it has been suggested, are reliable and valid measures 
for the assessment of adult reading capabilities. There seems to be a 
substantial positive relationship between a subject’s ability to use 
context clues » his ability to comprehend rapidly, and his general 
intelligence— particularly verbal intelligence. Supporting the 
methodological investigations involving children, the adult studies 
have shown that deleted words may be equally res tor able at ratios 
greater than 1:12. However, deletions more closely spaced than five 
words, tend to greatly Increase the difficulty of restoring the exact 
word deleted.. Cloze scores seem to Increase moderately with the 
decreasing density of deletions. While form class similarities seem 
tcT depend on the immediate verbal environment of a deletion, it is 
suggested that verbatim word predictability depends also upon the 
remote topical content or semantic features of the discourse in 
question. Cloze scores seem to correlate positively and significantly 

with the standardized measures used when exact— word scoring techniques 
are employed. 

A mechanical deletion method seems to be superior to any other 
technique. It has been pointed out that rational deletion ceases to 
measure passage difficulty, since the deletions are no longer 
representative of possible deletions within the sample. 

The relation of "sensitivity to literary style" and cloze scores 
has been investigated. It was indicated that word patterns of the 
material written in active verb form were more predictable, and 
presumably easier to comprehend, than those of nominalized sentences. 
However, the practice of using artificially— shortened sentences has 
been questioned, since comprehension as measured by cloze scores seems 
to be little effected by changes in sentence length. It is suggested 

that these results have implications for preparing instructional 
material. 

For the most part, investigations on information or knowledge gain 
by use of cloze tests have been unsuccessful. While the early work of 
Taylor (1954) suggests that knowledge gain can be measured validly and 
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reliably with the cloze technique, later work has questioned this 
conclusion (Coleman, 1966). Recent studies have argued that cloze 
procedure Is a useful technique for measuring the comprehension of 
oral messages under certain conditions. The need for additional 
Investigation In this area Is Indicated. It has been suggested that 
cloze procedures may not be a sensitive device when the vocabulary and 
concepts contained In a passage are more advanced than the knowledge or 
Information of the respondent would warrant. 

One feature of the cloze technique which contributes to Its general 
utility Is that It can be used to construct equivalent test forms drawn 
from the same or similar materials. It Is possible to construct 
multiple-test forms with similar means and variances and high 
Intercorrelatlons . For the classroom teacher the cloze procedure, then, 
has many potential uses. The teacher can readily determine the 
readability of passages to be used by children with particular abilities. 

It may be possible to obtain two cloze scores on the subject: 

(a) general comprehension skill and (b) specific comprehension relative 
to a particular subject. It might then be possible to examine the 
discrepancies between the two scores In order to provide suggestions 
for Individualizing teaching techniques and optimizing the use of time 
and materials. 

For the researcher, the cloze technique permits the study of some 
of the underlying processes Involved In reading. The use of Information 
gain or knowledge gain scores may permit the study of reading and 
listening In the Instructional process. The study of the Interrelation- 
ships between lexical and semantic comprehension might prove to be of 
practical value as well as theoretical value In the construction of 
Instructional material. 

CONCLUSIONS 

Research to date suggests that the most valid and reliable cloze 
test for measuring passage difficulty Is one In which: 

(1) An every nth mechanical mutilation system Is used. 

(2) Not more than 20 words out of every 100 are deleted. 

~ C3) Passage-len gth Is at least 250 words. 

(4) Deletion ratlosoFTn!0~MQ~±Td:2--in--longe r passages 

may be valid for certain purposes. ~ — -- 

(5) At least 50 words are deleted In order to Insure 
adequate sampling of passages. 

(6) The exact word deleted Is Indicated as the most useful 
and efficient scoring criteria. 

(7) Other scoring systems (synonym, form class) provide 
less Inter-scorer reliability and require substantially 
more time. 

(8) The separate scoring of form classes or content and 
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function words may provide specific Information for 
specialized purposes. 

Much cloze research leaves serious questions about the general- 
Izablllty of findings. Subject populations are not large and, seemingly, 
are selected for administrative reasons rather than by sound randomizing 
procedures. Information on subjects Is frequently lacking In detail. 

This Is particularly Important In studies of children. In some studies 
data on passage difficulty and cloze test Instructions are not reported. 
In other studies, test scores are reported on the subject’s reading 
achievement or I.Q. 

Implications of adult studies for children 

In studies of adult reading and cloze research, two variables are 
Infrequently mentioned: (a) ihe reading ability of the subject and 

(b) the difficulty of the passage (the original work of Taylor [1953, 
1954, 1957] and a recent study by Miller and Coleman [1966] are 
noteworthy exceptions to . It Is obvious that these two variables 
are of Importance In studies of reading and Instruction of children. 
Nevertheless, questions on cloze methodology Investigated with adult 
subjects are relevant to Investigations with children. 

For example, both Abom, et al. (1959) and MacGlnltle (1961) , 
working with adults, reported that context of less than four words 
between deletions substantially reduces contextual constraint and 
consequently excessively reduces the discrimination power of the cloze 
test. These findings were used by Bormuth (1962) as a basis for the 
construction of his cloze materials for use with children. However, 
Bormuth *s (1965a) data on optimum sample size In studies of children 
may be of questionable value when applied to adults. 

Investigations using adult subjects have supported the findings of 
studies with children In concluding that: 

(1) Consideration of synonyms and similarity of form class 
In scoring cloze responses or constructing cloze tests 
adds little If any discrimination power to the test 
but does raise test score means. 

(2) The Indiscriminate shortening of sentences has little 
effect on the comprehensibility of passages. 

Implications for additional research 

Little Investigation has been reported In several fundamental 
oxB&St Much work must be done considering the optimum methodology for 
assessing Individual reading comprehension for children. Exploration 
of differing deletion rates, passage lengths, and difficulty levels. 

In addition to separate form class scoring, should be considered. The 
measurement of Information gain may be possible under certain 
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circumstances through use of the cloze test. 

New areas in which the cloze procedure may be used Include the 
evaluation of the comprehension of instructional language. This research 
ma y include both oral and written presentation. The relationships 
between presentation length* frequency, difficulty, and student ability 
may be explored with cloze techniques. 

At least two components in reading comprehension may be considered; 
syntactic complexity and information load. In cloze research, these 
components have not been studied systematically. Assuming that they 
could be operationally defined, it may be possible to hold either or 
both of these variables constant in order to determine the relationships 
between cloze scores and these components and between the components 
themselves. While cloze scores may be reliable and valid measures of 
gross comprehension, it has yet to be shown that cloze scores are 
sensitive to either syntactic complexity or information load per se. 

One might hypothesize, for example, that cloze scores over function 
word deletions are more related to syntactic complexity than to information 
load. On the other hand, deleted content words may have more to do 
with information load than syntactic complexity. 

While certain significant investigations of cloze methods have 
contributed to their potential usefulness, much remains to be learned 
about the value of this relatively new technique in the study and use 
of language in the instructional setting. 



AUTHOR INDEX 



Abort! , M 


.3, 4, 16, 20, 


Hafner, L.E 


...6, 23, 




40, 41 


Harris, A.J 




Betts, E.A 


.7 


Jenklnson, M.E.... 


...5, 6 


Bloomer, R.H 


.16, 18, 32, 34 


Kingston, A.J 


...6, 23, 


Blumenfeld, J.P.. 


.29, 30 


Luke, M 




Bormuth, J.R 


.2. 4, 5, 7, 11, 


Louthan, V 






12, 13, 14, 15, 


MacGlnltle, W.H... 


...4, 19, 




17, 29, 31, 37, 


McLeod, J 


...11, 12 




40 


Miller, G.A 




Chall, J 


.5, 37 


Miller, G.R 




Coleman, E.B 


.7, 19, 20, 26, 


Musgrave, B 






29, 30, 32, 34 


Osgood, C.E 






39, 40 


Portnoy, S 




Dale, E 


.5, 37 


Rankin, E.F 


...4, 6, 


Darnell, D.K 


.29, 31, 32, 35 


Rubens teln, H 




Deutsch, M 


.8, 10 


Ruddell, P.B 


...2, 4, 


DeVito, J.A 


.35, 36 




12 


Dickens, M 


.32, 36 


Salzlnger, K 




Feldman, S 


.29, 30 


Schneyer, J.W 




Flllenbaum, S.... 


.23, 25 


Self ridge, J 




Fletcher, J.E.... 


.6, 7, 19, 22 


Spache, G 




Flesch, R 


.5 


Sterling, T.D 




Gallant, R 


.2, 8, 9 


Strickland , R. G. . . 




Greene, F.P 


.6, 26, 28 


Taylor, W.L 


...1. 2, 



44 



Taylor, W.L. (Cont.) . .7, 19, 21, 32, 

33, 35, 38, 40 

Thorndike, E.L 7 

Weaver, W.W 6, 23, 24, 29 

Wllllama, F 32, 36 



A5 



REFERENCES 

Aborn, M. , Rubensteln, H. , & Sterling, T.D. Sources of contextual 
constraint upon words in sentences* Journal of Experimental 
Peyohology, 1959, ^ (3), 171. 

Bennett, S., Semmel, M.I., & Barrett, L.S. In Lane, H.L. & Zale, E.N. 
(Eds.) Studies In language and language behavior. Center for 
Research on Language and Language Behavior, University of Michigan, 
1965, 414-428. 

Betts, E.A. Foundatione of reading instruction^ New York: American 
Book Company, 1946. 

Bloomer, R.H. The cloze procedure as a remedial reading exercise. 
Journal of Developmental Reading^ 1962, 173-181. 

Bloomer, R.H. & Heltzman, A.J. Pretesting and the efficiency of 
paragraph reading. Journal of Reading^ 1965, 8^ (4), 219. 

Bloomer, R.H. , Louthan, V., & Heltzman, A.J. Non-overt reinforced 
cloze procedure. U.S. Office of Education Project Report #2245, 
University of Connecticut, 1966. 

Blumenfeld, J.P. & Miller, G.R. Improving reading through teaching 
grammatical constraints. Elementary English^ 1966, 752. 

Bormuth, J.R. Cloze tests as measures of readability and comprehension 
ability. Unpublished doctoral dissertation. University of Indiana, 
1962. 

Bormuth, J.R. Experimental applications of cloze tests. International 
Reading Association Conference Proceedings^ 1964, 303. (a) 

Bormuth, J.R. Mean word depth as a predictor of comprehension 

difficulty. California Journal of Educational Research^ 1964, 13 (5), 
226. (b) 

Bormuth, J.R. Relationships between selected language variables and 
comprehension ability and difficulty. Cooperative Research Project 
#2082, U.S. Office of Education, 1964. (c) 

Bormuth, J.R. Optimum sample size and cloze test length in readability 
measurement. Journal of Educational Measurement^ 1965, 2^ (1), 111. 

(a) 

Bormuth, J.R. Validities of grammatical and semantic classifications 
of cloze scores. International Reading Association Conference 
Proceedings 9 1965, 283. (b) 




46 



Bormuth, J.R. & MacDoiiald, O.L. Cloze tests ss s messure of sblllty 
to detect literary style. In J. Allen Flgurel (Ed.), Reading and 
Inquiry t International Reading Aaeooiation Conference Prooeedinga, 

1965, 1£, 287-290. (c) 

Bormuth, J.R. Resdsblllty: A new spprosch. Reading Reaearoh Quarterly, 

1966, 1 (3), 79. (s) 

Bormuth, J.R. Design of readsblllty research. International Reading 
Aaeooiation Conferenoe Prooeedinga, 1966, 485-489. (b) 

Bormuth, J.R. Comparable cloze and multiple-choice comprehension 
test scores. Journal of Reading, 1967, 10, 291. (a) 

Bormuth, J.R. The Implications and use of cloze procedure In the 
evaluation of Instructional programs. Center for the Evaluation of 
Instructional Programs, University of California, Los Angeles, 

1967, (b) 

Bormuth, J.R. Cloze readability: Criterion reference scores. Journal 
of Educational Meaaurement, 1968 (In press). 

Carterette, E.C. & Jones, M.H. Redundancy In children's tests. 

Science, 1963, 140 . 1309-1311. 

Coleman, E.B. Improving comprehension by shortening sentences. 

Journal of Applied Payohology, 1962, 131-134. 

Coleman, E.B. & Blumenfeld, J.P. Cloze scores of nomlnallzatlons and 
their grammatical transformations using active verbs. Payohologioal 
Reporta, 1963, 13, 651-654. 

Coleman, E.B. Developing a technology of written Instruction: Some 
determiners of the complexity of prose. Paper read at symposium on 
Verbal Learning and Written Instruction, New York, March, 1966. 

Coleman, E.B. & Miller, G.R. A measure of Information gained during 
prose learning. Quarterly Journal of Reading, In press. 

Dale, E. & Chall, J. A formula for predicting readability. Educational 
Reaearoh Bulletin, January, 1948, 11-28. 

Dale, E. & Seels, B. Readability and reading. International Reading 
Aaeooiation Conferenoe Proceedinga, 1966. 

Darnell, D.K. The relation between sentence order and comprehension. 
Speech Monographa, 1963 , 30 , 97 . 

Deutsch, M. , Cherry, E. , Mallver, H. , & Brown, R. Communication for 
Information In the elementary classroom. Institute for Developmental 






47 



Studies, New York University, 1964. 

DeVito, J.A. Comprehension factors In oral and written discourse of 
skilled communicators. Speech Monographs , 1965, 32, 124. 

Dickens, M. & Williams, F. An experimental application of cloze 
procedure and attitude measures to listening comprehension. Speech 
Monographs, 1964, 3[i» 103-108. 

Ervin, S.M. Changes with age in the verbal determinants of word 
association. American Journal of Psychology, 1961, 74, 361-372. 

Epstein, W. The Influence of syntactical structure on learning. 

American Journal of Psychology, 1961, 74, 80-85. 

Flllenbaum, S. , Jones, L.V. , & Rapoport, A. The predictability of 
words and their grammatical classes as a function of rate of deletion 
from a speech transcript. Journal of Verbal Learning and Verbal 
Behavior, 1963, 2^ 186-194. 

Fletcher, J.E. A study of the relationships between ability to use 
context as an aid in reading and other verbal abilities. Unpublished 
doctoral dissertation. University of Washington, 1959. 

Flesch, R. The art of readable writing. Harper & Row^ 1949. 

Gallant, R. Use of cloze tests as a measure of readability in the 
primary grades. Proceedings of the International Reading Association 
Convention, 1965, 10, 286-287. 

Greene, F.P. A modified cloze procedure for assessing adult reading 
comprehension. Unpublished doctoral dissertation. University of 
Michigan, 1964. 

Greene, F.P. Modification of the cloze procedure and changes in 

reading test performances. Journal of Educational Measurement, 1965, \ 

2 (2), 213-217.' 

Hafner, L.E. Relationships of various measures of the cloze. In E.L. 
Thurston & L.E. Hafner (Eds.) Thirteenth Yearbook of the National 
Reading Conference, Milwaukee, Wisconsin, National Reading Conference, 
Inc., 1963, 135-145. 

Hafner, L.E. Implications of cloze. In E.L. Thurston & L.E. Hafner 
(Eds.) The philosophical and sociological bases of reading. 

Fourteenth Yearbook of the National Reading Conference, Milwaukee, 
Wisconsin, The National Reading Conference, Inc., 1965, 151-158. 

Hafner, L.E. Cloze procedure. Journal of Reading, 1966, 9^, 415-421. 



48 



Harris, A.J. Effective teaching of reading. New York: David McKay, 
1962. 

Jenklnson, M.E. Selected processes and difficulties in reading 
comprehension. Unpublished doctoral dissertation. University of 
Chicago, 1957. 

Kerfoot, J.F. Reading in the elementary school. Reviet/3 of Educational 
Research^ 1967 , 120. 

Kingston, A.ji & Weaver, W.W. Recent development in readability 
appraisal. Journal of Reading ^ 1967, 44. 



Klare, G.R. The measurement of readability. Am< ate 

University - 



Louthan, V. ons and their effects 

on reading 965, 295. 

Luke, M. Form class and cloze procedure. Unpublished manuscript. 

Language Development Program, Center for Human Growth and Develoiment , 
University of Michigan, 1964. 

MacGlnltie, H.H. Contextual constraint in English prose paragraphs. 
Joumdl of Psychology, 1961, 51, 121-130. 

Marks, M.R. & Taylor, W.L. The Influence of contextual and goal 
constraints on the meaningfulness of "automatic sentences". 

Journal of Social Psychology, 1954, 43-51. 

McLeod, J. GAP reading comprehension test. Melbourne, Australia: 
Helnemann, 1965. 

McLeod, J. & Anderson, J. Readability assessment and word redundancy 
of printed English. Psychological Reports, 1966, 18, 35-38. 

Miller, G.A. & Friedman, E.A. The reconstruction of mutilated English 
texts. Information and Control, 1957, 1, 38-55. 

Miller, G.A. & Self ridge, J. Verbal context and the recall of meaningful 
material. American Journal of Psychology, 1950, 176-185. 

Miller, G.R. & Coleman, E.B. A set of 36 passages calibrated for 
comprehensibility. Inglewood, California: Southwest Regional 
Laboratory for Educational Research and Development, 1966. 

Musgrave, B. The effect of who and what context on cloze and 




Lee, H.D. W 
Journal of 




tell the classroom teacher? 



A9 



commonality scores. Journal of Social Peychology^ 1963 » 185- 

192. 

Osgood, C.E. The nature and measurement of meaning. Peyohologioal 
Bulletin^ 1962, ^ (3), 197-237. 

Rankin, E.F. An evaluation of the cloze procedure as a technique for 
measuring reading comprehension. Unpublished doctoral dissertation. 
University of Michigan, 1957. 

Rankin, E.F. Uses of the cloze procedure In the reading clinic. In 
J.A. Flgurel (Ed.), IRA Conference Proceedings, New York. 

Soholaetic Magazine^ 1959, 228-232. 

Rankin, E.F. The cloze procedure — Its validity and utility. In O.S. 
Causey and W. Eller (Eds.), Eighth Yearbook of the National Reading 
Conference^ National Reading Conference, Inc., 1959, 8, 131-144. 

Rankin, E.F. The cloze procedure — A survey of research. In E.L. 

Thurston and L.E. Hafner (Eds.) The philoeophical and sociological 
bases of reading. Fourteenth Yearbook of the National Reading 
Conference, Milwaukee, Wisconsin, The National Reading Conference, 

Inc., 1965, 133-150. 

Rankin, E.F. Residual gain as a measure of Individual differences In 
reading Improvement. Journal of Reading ^ 1965, 224-233. 

Rankin, E.F. Research design and the cloze procedure. Proceedings of 
the International Reading Association Convention, 1966, ^ (1), 489- 
491. 

Ruddell, R.B. The effect of oral and written patterns of language 

structure on reading comprehension. Unpublished doctoral dissertation. 
University of Indiana, 1963. 

Ruddell, R.B. A study of the cloze comprehension technique In relation 
to structurally controlled reading material. In J.A. Flgurel (Ed.) 
Improvement of reading through classroom practice. International 
Reading Association Conference Proceedings, 1964, 298-306. 

Ruddell, R.B. The effect of oral and written patterns of language . 
structure on reading comprehension. Reading Teacher^ 1965, 270. 

Salzlhger, K. , Portnoy, S., & Feldman, R.S. The effect of order of 
approxU^tlon to the statistical structure of English on the 
emission of verbal responses. Journal of Experimental Psychology, 

1962, (^, 52-57. 

Schneyer, J.W. Use of the cloze procedure for Improving reading 
comprehension. Reading Teacher, 1965, 174-179. 



50 



Spache, G. A new readability formula for primary grade materials. 
Elementary School Journal^ 1952, ^ (7), 410-413. 

Strickland, R.G. The language of elementary school children; Its 

relationship to the language of reading textbooks and the quality of 

reading of selected children. Bulletin of the School of Education^ 
University of Indiana, 1962, ^ (4). 

Taylor, W.L. Cloze procedure; A new tool for measuring readability. 
Joumaliam Quarterly ^ 1953, 414-438. 

Taylor, W.L. Application of ''cloze” and entropy measures to the study 
of contextual constraint in samples of continuous prose. Unpublished 
doctoral dissertation. University of Illinois, 1954. 

Taylor, W.L. Recent developments in the use of the cloze procedure. 
Joumalim Quarterly , 1956, 42-48. 

Taylor, W.L. Cloze readability scores as indices of individual 
differences in comprehension and aptitude. Journal of Applied 

Peychologys 1957, 19-26. 

Thompson, E. Some effects of message structure on listeners 
comprehension. Speech Monographs^ 1957 , _24, 51. 

Thorndike, E.L. Reading and reasoning; A study of mistakes in 
paragraph reading. Journal of Educational Psychology » 1917, 8, 
323-332. 

Tomlinson, ''l.N. Sources of contextual constraint upon words in 
sentences. Unpublished masters thesis. University of California, 

Los Angeles, 1967. 

Weaver, W.W. The predictibility of omissions in reading and listening. 
Eleventh Yearbook of the National Reading Conference, Milwaukee, 
Wisconsin, 1962, 148-153. 

Weaver, W.W. Theoretical aspects of the cloze procedure. In E.L. 
Thurston and L.E. Hafner (Eds.) The philosophical and sociological 
bases of x*eadiiiy» Fourteenth Yearbook of the National Reading 
Conference, Milwaukee, Wisconsin, The National Reading Conference, 
1965, 115-132. 

Weaver, W.W. & Kingston, A.J. A factor analysis of the cloze procedure 
and other measures of reading and language ability. Journal of 
Communication, 1963, 13, 252-261. 



